Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/cp3d_ndu_avg_feats_no_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/cp3d_ndu_avg_feats_no_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=1),
 'dropout': 0.4,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': True,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_avg/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 1000,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 1000,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'metrics': ['binary_cross_entropy'],
 'minimize_score': True,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/cp3d_ndu_avg_feats_no_mpnn_from_binary_cross_entropy',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_avg/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_avg/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 1,303,001
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.306016
Epoch 1
Validation binary_cross_entropy = 0.532507
Epoch 2
Validation binary_cross_entropy = 0.857168
Epoch 3
Validation binary_cross_entropy = 0.366425
Epoch 4
Loss = 1.0041e+00, PNorm = 38.3088, GNorm = 11.1213, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.417989
Epoch 5
Validation binary_cross_entropy = 0.822771
Epoch 6
Validation binary_cross_entropy = 0.406706
Epoch 7
Validation binary_cross_entropy = 0.312841
Epoch 8
Validation binary_cross_entropy = 0.780055
Epoch 9
Loss = 7.5409e-01, PNorm = 38.3932, GNorm = 8.7386, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.756440
Epoch 10
Validation binary_cross_entropy = 0.344387
Epoch 11
Validation binary_cross_entropy = 0.392850
Epoch 12
Validation binary_cross_entropy = 0.424813
Epoch 13
Validation binary_cross_entropy = 0.314390
Epoch 14
Loss = 9.3789e-01, PNorm = 38.5048, GNorm = 6.9389, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.320299
Epoch 15
Validation binary_cross_entropy = 0.534103
Epoch 16
Validation binary_cross_entropy = 0.369272
Epoch 17
Validation binary_cross_entropy = 0.301588
Epoch 18
Validation binary_cross_entropy = 0.429998
Epoch 19
Loss = 5.4898e-01, PNorm = 38.6108, GNorm = 5.6204, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.812773
Epoch 20
Validation binary_cross_entropy = 0.424292
Epoch 21
Validation binary_cross_entropy = 0.319324
Epoch 22
Validation binary_cross_entropy = 0.638530
Epoch 23
Validation binary_cross_entropy = 0.702932
Epoch 24
Loss = 5.5699e-01, PNorm = 38.7021, GNorm = 4.0511, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.294836
Epoch 25
Validation binary_cross_entropy = 0.308393
Epoch 26
Validation binary_cross_entropy = 0.564045
Epoch 27
Validation binary_cross_entropy = 0.571008
Epoch 28
Validation binary_cross_entropy = 0.298190
Epoch 29
Loss = 6.8301e-01, PNorm = 38.7766, GNorm = 6.6031, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.307567
Epoch 30
Validation binary_cross_entropy = 0.331520
Epoch 31
Validation binary_cross_entropy = 0.438962
Epoch 32
Validation binary_cross_entropy = 0.487848
Epoch 33
Validation binary_cross_entropy = 0.427553
Epoch 34
Loss = 5.3174e-01, PNorm = 38.8463, GNorm = 5.3473, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.388180
Epoch 35
Validation binary_cross_entropy = 0.409366
Epoch 36
Validation binary_cross_entropy = 0.446507
Epoch 37
Validation binary_cross_entropy = 0.417857
Epoch 38
Validation binary_cross_entropy = 0.307553
Epoch 39
Loss = 6.2863e-01, PNorm = 38.9084, GNorm = 5.2646, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.333253
Epoch 40
Validation binary_cross_entropy = 0.403811
Epoch 41
Validation binary_cross_entropy = 0.455199
Epoch 42
Validation binary_cross_entropy = 0.455982
Epoch 43
Validation binary_cross_entropy = 0.387855
Epoch 44
Loss = 5.8846e-01, PNorm = 38.9646, GNorm = 4.7552, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.361012
Epoch 45
Validation binary_cross_entropy = 0.471129
Epoch 46
Validation binary_cross_entropy = 0.447366
Epoch 47
Validation binary_cross_entropy = 0.423029
Epoch 48
Validation binary_cross_entropy = 0.427930
Epoch 49
Loss = 6.1356e-01, PNorm = 39.0165, GNorm = 4.7255, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.452636
Epoch 50
Validation binary_cross_entropy = 0.440496
Epoch 51
Validation binary_cross_entropy = 0.393918
Epoch 52
Validation binary_cross_entropy = 0.490571
Epoch 53
Validation binary_cross_entropy = 0.630071
Epoch 54
Loss = 5.8288e-01, PNorm = 39.0673, GNorm = 3.5256, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.547764
Epoch 55
Validation binary_cross_entropy = 0.460439
Epoch 56
Validation binary_cross_entropy = 0.430667
Epoch 57
Validation binary_cross_entropy = 0.430381
Epoch 58
Validation binary_cross_entropy = 0.514795
Epoch 59
Loss = 4.8724e-01, PNorm = 39.1158, GNorm = 3.7107, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.585988
Epoch 60
Validation binary_cross_entropy = 0.599981
Epoch 61
Validation binary_cross_entropy = 0.530317
Epoch 62
Validation binary_cross_entropy = 0.458923
Epoch 63
Validation binary_cross_entropy = 0.447314
Epoch 64
Loss = 4.8811e-01, PNorm = 39.1641, GNorm = 3.9246, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.508000
Epoch 65
Validation binary_cross_entropy = 0.594114
Epoch 66
Validation binary_cross_entropy = 0.546738
Epoch 67
Validation binary_cross_entropy = 0.526190
Epoch 68
Validation binary_cross_entropy = 0.597792
Epoch 69
Loss = 5.4533e-01, PNorm = 39.2091, GNorm = 3.1967, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.628878
Epoch 70
Validation binary_cross_entropy = 0.614347
Epoch 71
Validation binary_cross_entropy = 0.725046
Epoch 72
Validation binary_cross_entropy = 0.740357
Epoch 73
Validation binary_cross_entropy = 0.555696
Epoch 74
Loss = 5.5502e-01, PNorm = 39.2499, GNorm = 3.4724, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.490953
Epoch 75
Validation binary_cross_entropy = 0.508246
Epoch 76
Validation binary_cross_entropy = 0.537006
Epoch 77
Validation binary_cross_entropy = 0.558339
Epoch 78
Validation binary_cross_entropy = 0.514621
Epoch 79
Loss = 4.5097e-01, PNorm = 39.2969, GNorm = 3.1314, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.458172
Epoch 80
Validation binary_cross_entropy = 0.423135
Epoch 81
Validation binary_cross_entropy = 0.417067
Epoch 82
Validation binary_cross_entropy = 0.452022
Epoch 83
Validation binary_cross_entropy = 0.530892
Epoch 84
Loss = 5.1718e-01, PNorm = 39.3418, GNorm = 2.5516, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.501414
Epoch 85
Validation binary_cross_entropy = 0.419528
Epoch 86
Validation binary_cross_entropy = 0.377027
Epoch 87
Validation binary_cross_entropy = 0.375425
Epoch 88
Validation binary_cross_entropy = 0.447538
Epoch 89
Loss = 5.1243e-01, PNorm = 39.3865, GNorm = 3.8592, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.531835
Epoch 90
Validation binary_cross_entropy = 0.478648
Epoch 91
Validation binary_cross_entropy = 0.403040
Epoch 92
Validation binary_cross_entropy = 0.409122
Epoch 93
Validation binary_cross_entropy = 0.402888
Epoch 94
Loss = 4.2593e-01, PNorm = 39.4314, GNorm = 2.7912, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.391430
Epoch 95
Validation binary_cross_entropy = 0.433710
Epoch 96
Validation binary_cross_entropy = 0.407703
Epoch 97
Validation binary_cross_entropy = 0.367332
Epoch 98
Validation binary_cross_entropy = 0.437936
Epoch 99
Loss = 5.1936e-01, PNorm = 39.4748, GNorm = 2.3406, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.482798
Epoch 100
Validation binary_cross_entropy = 0.505179
Epoch 101
Validation binary_cross_entropy = 0.497676
Epoch 102
Validation binary_cross_entropy = 0.446610
Epoch 103
Validation binary_cross_entropy = 0.393697
Epoch 104
Loss = 4.7416e-01, PNorm = 39.5208, GNorm = 2.9830, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.411027
Epoch 105
Validation binary_cross_entropy = 0.471223
Epoch 106
Validation binary_cross_entropy = 0.547324
Epoch 107
Validation binary_cross_entropy = 0.552724
Epoch 108
Validation binary_cross_entropy = 0.476846
Epoch 109
Loss = 4.2303e-01, PNorm = 39.5604, GNorm = 2.4472, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.399456
Epoch 110
Validation binary_cross_entropy = 0.413632
Epoch 111
Validation binary_cross_entropy = 0.592264
Epoch 112
Validation binary_cross_entropy = 0.601528
Epoch 113
Validation binary_cross_entropy = 0.463385
Epoch 114
Loss = 3.9770e-01, PNorm = 39.6026, GNorm = 2.1050, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.395607
Epoch 115
Validation binary_cross_entropy = 0.375896
Epoch 116
Validation binary_cross_entropy = 0.395226
Epoch 117
Validation binary_cross_entropy = 0.436520
Epoch 118
Validation binary_cross_entropy = 0.465898
Epoch 119
Loss = 3.8446e-01, PNorm = 39.6476, GNorm = 2.4469, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.507582
Epoch 120
Validation binary_cross_entropy = 0.499194
Epoch 121
Validation binary_cross_entropy = 0.422272
Epoch 122
Validation binary_cross_entropy = 0.388162
Epoch 123
Validation binary_cross_entropy = 0.421077
Epoch 124
Loss = 3.5034e-01, PNorm = 39.6918, GNorm = 3.2939, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.440107
Epoch 125
Validation binary_cross_entropy = 0.526206
Epoch 126
Validation binary_cross_entropy = 0.488835
Epoch 127
Validation binary_cross_entropy = 0.461081
Epoch 128
Validation binary_cross_entropy = 0.454975
Epoch 129
Loss = 3.5195e-01, PNorm = 39.7352, GNorm = 3.6700, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.422957
Epoch 130
Validation binary_cross_entropy = 0.410212
Epoch 131
Validation binary_cross_entropy = 0.416230
Epoch 132
Validation binary_cross_entropy = 0.458490
Epoch 133
Validation binary_cross_entropy = 0.551913
Epoch 134
Loss = 3.7983e-01, PNorm = 39.7764, GNorm = 3.1925, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.549997
Epoch 135
Validation binary_cross_entropy = 0.504552
Epoch 136
Validation binary_cross_entropy = 0.438151
Epoch 137
Validation binary_cross_entropy = 0.409831
Epoch 138
Validation binary_cross_entropy = 0.417039
Epoch 139
Loss = 3.4916e-01, PNorm = 39.8183, GNorm = 2.8528, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.449378
Epoch 140
Validation binary_cross_entropy = 0.424510
Epoch 141
Validation binary_cross_entropy = 0.431424
Epoch 142
Validation binary_cross_entropy = 0.428291
Epoch 143
Validation binary_cross_entropy = 0.417636
Epoch 144
Loss = 4.4306e-01, PNorm = 39.8545, GNorm = 4.3958, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.412144
Epoch 145
Validation binary_cross_entropy = 0.503587
Epoch 146
Validation binary_cross_entropy = 0.592396
Epoch 147
Validation binary_cross_entropy = 0.485389
Epoch 148
Validation binary_cross_entropy = 0.429399
Epoch 149
Loss = 3.0002e-01, PNorm = 39.8927, GNorm = 2.0267, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.473686
Epoch 150
Validation binary_cross_entropy = 0.481882
Epoch 151
Validation binary_cross_entropy = 0.474789
Epoch 152
Validation binary_cross_entropy = 0.481823
Epoch 153
Validation binary_cross_entropy = 0.450859
Epoch 154
Loss = 4.6860e-01, PNorm = 39.9322, GNorm = 3.0097, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.406193
Epoch 155
Validation binary_cross_entropy = 0.392884
Epoch 156
Validation binary_cross_entropy = 0.420302
Epoch 157
Validation binary_cross_entropy = 0.448594
Epoch 158
Validation binary_cross_entropy = 0.460907
Epoch 159
Loss = 4.1559e-01, PNorm = 39.9718, GNorm = 2.9619, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.420942
Epoch 160
Validation binary_cross_entropy = 0.388093
Epoch 161
Validation binary_cross_entropy = 0.371936
Epoch 162
Validation binary_cross_entropy = 0.404847
Epoch 163
Validation binary_cross_entropy = 0.431059
Epoch 164
Loss = 4.7447e-01, PNorm = 40.0123, GNorm = 3.3367, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.493851
Epoch 165
Validation binary_cross_entropy = 0.588207
Epoch 166
Validation binary_cross_entropy = 0.554958
Epoch 167
Validation binary_cross_entropy = 0.431838
Epoch 168
Validation binary_cross_entropy = 0.366635
Epoch 169
Loss = 3.7353e-01, PNorm = 40.0527, GNorm = 4.3882, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.386229
Epoch 170
Validation binary_cross_entropy = 0.463800
Epoch 171
Validation binary_cross_entropy = 0.485806
Epoch 172
Validation binary_cross_entropy = 0.444762
Epoch 173
Validation binary_cross_entropy = 0.436004
Epoch 174
Loss = 4.0061e-01, PNorm = 40.0933, GNorm = 2.5907, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.422285
Epoch 175
Validation binary_cross_entropy = 0.436126
Epoch 176
Validation binary_cross_entropy = 0.434285
Epoch 177
Validation binary_cross_entropy = 0.417497
Epoch 178
Validation binary_cross_entropy = 0.400259
Epoch 179
Loss = 5.3427e-01, PNorm = 40.1301, GNorm = 3.2820, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.397955
Epoch 180
Validation binary_cross_entropy = 0.418429
Epoch 181
Validation binary_cross_entropy = 0.453386
Epoch 182
Validation binary_cross_entropy = 0.489103
Epoch 183
Validation binary_cross_entropy = 0.553841
Epoch 184
Loss = 3.6578e-01, PNorm = 40.1659, GNorm = 2.4643, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.578517
Epoch 185
Validation binary_cross_entropy = 0.470285
Epoch 186
Validation binary_cross_entropy = 0.401492
Epoch 187
Validation binary_cross_entropy = 0.407752
Epoch 188
Validation binary_cross_entropy = 0.425408
Epoch 189
Loss = 3.0927e-01, PNorm = 40.2011, GNorm = 2.6494, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.466552
Epoch 190
Validation binary_cross_entropy = 0.452295
Epoch 191
Validation binary_cross_entropy = 0.429437
Epoch 192
Validation binary_cross_entropy = 0.412010
Epoch 193
Validation binary_cross_entropy = 0.406836
Epoch 194
Loss = 3.9496e-01, PNorm = 40.2321, GNorm = 2.6693, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.422288
Epoch 195
Validation binary_cross_entropy = 0.454788
Epoch 196
Validation binary_cross_entropy = 0.547720
Epoch 197
Validation binary_cross_entropy = 0.586336
Epoch 198
Validation binary_cross_entropy = 0.535450
Epoch 199
Loss = 4.2669e-01, PNorm = 40.2638, GNorm = 2.3965, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.473896
Epoch 200
Validation binary_cross_entropy = 0.441726
Epoch 201
Validation binary_cross_entropy = 0.440854
Epoch 202
Validation binary_cross_entropy = 0.480715
Epoch 203
Validation binary_cross_entropy = 0.504497
Epoch 204
Loss = 3.2913e-01, PNorm = 40.2977, GNorm = 2.8351, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.462636
Epoch 205
Validation binary_cross_entropy = 0.445886
Epoch 206
Validation binary_cross_entropy = 0.401656
Epoch 207
Validation binary_cross_entropy = 0.387562
Epoch 208
Validation binary_cross_entropy = 0.379101
Epoch 209
Loss = 3.2088e-01, PNorm = 40.3325, GNorm = 2.4600, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.414364
Epoch 210
Validation binary_cross_entropy = 0.467272
Epoch 211
Validation binary_cross_entropy = 0.508036
Epoch 212
Validation binary_cross_entropy = 0.511708
Epoch 213
Validation binary_cross_entropy = 0.404108
Epoch 214
Loss = 3.0055e-01, PNorm = 40.3685, GNorm = 3.1277, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.361771
Epoch 215
Validation binary_cross_entropy = 0.380731
Epoch 216
Validation binary_cross_entropy = 0.430562
Epoch 217
Validation binary_cross_entropy = 0.476348
Epoch 218
Validation binary_cross_entropy = 0.458287
Epoch 219
Loss = 2.8655e-01, PNorm = 40.4094, GNorm = 1.8799, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.439848
Epoch 220
Validation binary_cross_entropy = 0.417693
Epoch 221
Validation binary_cross_entropy = 0.413584
Epoch 222
Validation binary_cross_entropy = 0.404134
Epoch 223
Validation binary_cross_entropy = 0.396411
Epoch 224
Loss = 2.8655e-01, PNorm = 40.4457, GNorm = 1.9784, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.377613
Epoch 225
Validation binary_cross_entropy = 0.381462
Epoch 226
Validation binary_cross_entropy = 0.465343
Epoch 227
Validation binary_cross_entropy = 0.408768
Epoch 228
Validation binary_cross_entropy = 0.400433
Epoch 229
Loss = 3.1289e-01, PNorm = 40.4806, GNorm = 2.5581, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.394286
Epoch 230
Validation binary_cross_entropy = 0.364369
Epoch 231
Validation binary_cross_entropy = 0.351921
Epoch 232
Validation binary_cross_entropy = 0.362642
Epoch 233
Validation binary_cross_entropy = 0.363065
Epoch 234
Loss = 2.9878e-01, PNorm = 40.5182, GNorm = 2.0422, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.369927
Epoch 235
Validation binary_cross_entropy = 0.382206
Epoch 236
Validation binary_cross_entropy = 0.387649
Epoch 237
Validation binary_cross_entropy = 0.373227
Epoch 238
Validation binary_cross_entropy = 0.352287
Epoch 239
Loss = 3.3333e-01, PNorm = 40.5529, GNorm = 2.1116, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.338753
Epoch 240
Validation binary_cross_entropy = 0.346896
Epoch 241
Validation binary_cross_entropy = 0.370773
Epoch 242
Validation binary_cross_entropy = 0.414969
Epoch 243
Validation binary_cross_entropy = 0.400043
Epoch 244
Loss = 3.1282e-01, PNorm = 40.5858, GNorm = 3.4541, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.380210
Epoch 245
Validation binary_cross_entropy = 0.361525
Epoch 246
Validation binary_cross_entropy = 0.361100
Epoch 247
Validation binary_cross_entropy = 0.384305
Epoch 248
Validation binary_cross_entropy = 0.425223
Epoch 249
Loss = 4.8428e-01, PNorm = 40.6178, GNorm = 3.7876, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.440900
Epoch 250
Validation binary_cross_entropy = 0.415096
Epoch 251
Validation binary_cross_entropy = 0.386752
Epoch 252
Validation binary_cross_entropy = 0.376772
Epoch 253
Validation binary_cross_entropy = 0.382018
Epoch 254
Loss = 3.1452e-01, PNorm = 40.6466, GNorm = 2.0021, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.397883
Epoch 255
Validation binary_cross_entropy = 0.406631
Epoch 256
Validation binary_cross_entropy = 0.424097
Epoch 257
Validation binary_cross_entropy = 0.437042
Epoch 258
Validation binary_cross_entropy = 0.441189
Epoch 259
Loss = 2.9189e-01, PNorm = 40.6764, GNorm = 1.7452, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.430992
Epoch 260
Validation binary_cross_entropy = 0.400179
Epoch 261
Validation binary_cross_entropy = 0.363437
Epoch 262
Validation binary_cross_entropy = 0.364886
Epoch 263
Validation binary_cross_entropy = 0.373093
Epoch 264
Loss = 4.4172e-01, PNorm = 40.7067, GNorm = 3.6665, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.354426
Epoch 265
Validation binary_cross_entropy = 0.356000
Epoch 266
Validation binary_cross_entropy = 0.367056
Epoch 267
Validation binary_cross_entropy = 0.389506
Epoch 268
Validation binary_cross_entropy = 0.413349
Epoch 269
Loss = 3.8611e-01, PNorm = 40.7376, GNorm = 3.3949, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.432019
Epoch 270
Validation binary_cross_entropy = 0.439066
Epoch 271
Validation binary_cross_entropy = 0.420955
Epoch 272
Validation binary_cross_entropy = 0.409281
Epoch 273
Validation binary_cross_entropy = 0.378899
Epoch 274
Loss = 3.3712e-01, PNorm = 40.7704, GNorm = 3.6577, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.374509
Epoch 275
Validation binary_cross_entropy = 0.381112
Epoch 276
Validation binary_cross_entropy = 0.387107
Epoch 277
Validation binary_cross_entropy = 0.376844
Epoch 278
Validation binary_cross_entropy = 0.346465
Epoch 279
Loss = 3.2119e-01, PNorm = 40.8035, GNorm = 3.6224, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.334405
Epoch 280
Validation binary_cross_entropy = 0.343923
Epoch 281
Validation binary_cross_entropy = 0.357075
Epoch 282
Validation binary_cross_entropy = 0.389097
Epoch 283
Validation binary_cross_entropy = 0.425493
Epoch 284
Loss = 3.3830e-01, PNorm = 40.8349, GNorm = 2.9265, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.391686
Epoch 285
Validation binary_cross_entropy = 0.341581
Epoch 286
Validation binary_cross_entropy = 0.308861
Epoch 287
Validation binary_cross_entropy = 0.297015
Epoch 288
Validation binary_cross_entropy = 0.301453
Epoch 289
Loss = 4.2762e-01, PNorm = 40.8683, GNorm = 3.3779, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.320063
Epoch 290
Validation binary_cross_entropy = 0.356938
Epoch 291
Validation binary_cross_entropy = 0.386373
Epoch 292
Validation binary_cross_entropy = 0.402998
Epoch 293
Validation binary_cross_entropy = 0.408480
Epoch 294
Loss = 2.8777e-01, PNorm = 40.8979, GNorm = 1.6518, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.394425
Epoch 295
Validation binary_cross_entropy = 0.412654
Epoch 296
Validation binary_cross_entropy = 0.455400
Epoch 297
Validation binary_cross_entropy = 0.411346
Epoch 298
Validation binary_cross_entropy = 0.397634
Epoch 299
Loss = 4.1317e-01, PNorm = 40.9280, GNorm = 2.4900, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.395018
Model 0 best validation binary_cross_entropy = 0.294836 on epoch 24
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.196439
Ensemble test binary_cross_entropy = 0.196439
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 1,303,001
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.319132
Epoch 1
Validation binary_cross_entropy = 0.569873
Epoch 2
Validation binary_cross_entropy = 0.520094
Epoch 3
Validation binary_cross_entropy = 0.389554
Epoch 4
Loss = 8.6068e-01, PNorm = 38.3075, GNorm = 8.9629, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.489238
Epoch 5
Validation binary_cross_entropy = 0.427411
Epoch 6
Validation binary_cross_entropy = 0.413654
Epoch 7
Validation binary_cross_entropy = 0.353288
Epoch 8
Validation binary_cross_entropy = 0.315903
Epoch 9
Loss = 9.4460e-01, PNorm = 38.3981, GNorm = 8.1811, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.442592
Epoch 10
Validation binary_cross_entropy = 0.443983
Epoch 11
Validation binary_cross_entropy = 0.377420
Epoch 12
Validation binary_cross_entropy = 0.384045
Epoch 13
Validation binary_cross_entropy = 0.358180
Epoch 14
Loss = 9.6627e-01, PNorm = 38.5171, GNorm = 8.9765, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.607640
Epoch 15
Validation binary_cross_entropy = 0.685004
Epoch 16
Validation binary_cross_entropy = 0.339288
Epoch 17
Validation binary_cross_entropy = 0.290353
Epoch 18
Validation binary_cross_entropy = 0.818409
Epoch 19
Loss = 5.7411e-01, PNorm = 38.6218, GNorm = 4.4683, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.877149
Epoch 20
Validation binary_cross_entropy = 0.418441
Epoch 21
Validation binary_cross_entropy = 0.422370
Epoch 22
Validation binary_cross_entropy = 0.408678
Epoch 23
Validation binary_cross_entropy = 0.534082
Epoch 24
Loss = 6.1344e-01, PNorm = 38.7136, GNorm = 5.6598, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.461328
Epoch 25
Validation binary_cross_entropy = 0.296961
Epoch 26
Validation binary_cross_entropy = 0.315855
Epoch 27
Validation binary_cross_entropy = 0.566285
Epoch 28
Validation binary_cross_entropy = 0.770931
Epoch 29
Loss = 7.2712e-01, PNorm = 38.7901, GNorm = 7.8907, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.477304
Epoch 30
Validation binary_cross_entropy = 0.294668
Epoch 31
Validation binary_cross_entropy = 0.364339
Epoch 32
Validation binary_cross_entropy = 0.646184
Epoch 33
Validation binary_cross_entropy = 0.522412
Epoch 34
Loss = 6.8188e-01, PNorm = 38.8582, GNorm = 5.3212, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.407261
Epoch 35
Validation binary_cross_entropy = 0.522157
Epoch 36
Validation binary_cross_entropy = 0.649209
Epoch 37
Validation binary_cross_entropy = 0.523612
Epoch 38
Validation binary_cross_entropy = 0.374217
Epoch 39
Loss = 6.6915e-01, PNorm = 38.9192, GNorm = 6.1315, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.353352
Epoch 40
Validation binary_cross_entropy = 0.573105
Epoch 41
Validation binary_cross_entropy = 0.953238
Epoch 42
Validation binary_cross_entropy = 0.860858
Epoch 43
Validation binary_cross_entropy = 0.540465
Epoch 44
Loss = 5.5453e-01, PNorm = 38.9729, GNorm = 4.3796, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.525753
Epoch 45
Validation binary_cross_entropy = 0.829420
Epoch 46
Validation binary_cross_entropy = 1.884655
Epoch 47
Validation binary_cross_entropy = 1.302799
Epoch 48
Validation binary_cross_entropy = 0.790679
Epoch 49
Loss = 5.4736e-01, PNorm = 39.0246, GNorm = 4.2292, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.500750
Epoch 50
Validation binary_cross_entropy = 0.449762
Epoch 51
Validation binary_cross_entropy = 0.578399
Epoch 52
Validation binary_cross_entropy = 0.754531
Epoch 53
Validation binary_cross_entropy = 0.694118
Epoch 54
Loss = 5.4019e-01, PNorm = 39.0737, GNorm = 4.5596, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.477189
Epoch 55
Validation binary_cross_entropy = 0.402771
Epoch 56
Validation binary_cross_entropy = 0.506842
Epoch 57
Validation binary_cross_entropy = 0.708750
Epoch 58
Validation binary_cross_entropy = 0.648340
Epoch 59
Loss = 5.0782e-01, PNorm = 39.1148, GNorm = 2.9778, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.551390
Epoch 60
Validation binary_cross_entropy = 0.495793
Epoch 61
Validation binary_cross_entropy = 0.451088
Epoch 62
Validation binary_cross_entropy = 0.452412
Epoch 63
Validation binary_cross_entropy = 0.479526
Epoch 64
Loss = 4.6323e-01, PNorm = 39.1562, GNorm = 2.5589, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.546010
Epoch 65
Validation binary_cross_entropy = 0.514524
Epoch 66
Validation binary_cross_entropy = 0.463085
Epoch 67
Validation binary_cross_entropy = 0.493333
Epoch 68
Validation binary_cross_entropy = 0.633827
Epoch 69
Loss = 5.4406e-01, PNorm = 39.1953, GNorm = 3.1942, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.647234
Epoch 70
Validation binary_cross_entropy = 0.640645
Epoch 71
Validation binary_cross_entropy = 0.612594
Epoch 72
Validation binary_cross_entropy = 0.601094
Epoch 73
Validation binary_cross_entropy = 0.514170
Epoch 74
Loss = 5.5978e-01, PNorm = 39.2347, GNorm = 4.0296, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.493121
Epoch 75
Validation binary_cross_entropy = 0.493365
Epoch 76
Validation binary_cross_entropy = 0.509653
Epoch 77
Validation binary_cross_entropy = 0.498211
Epoch 78
Validation binary_cross_entropy = 0.461338
Epoch 79
Loss = 4.7741e-01, PNorm = 39.2724, GNorm = 4.1189, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.491528
Epoch 80
Validation binary_cross_entropy = 0.493170
Epoch 81
Validation binary_cross_entropy = 0.414110
Epoch 82
Validation binary_cross_entropy = 0.372418
Epoch 83
Validation binary_cross_entropy = 0.414789
Epoch 84
Loss = 5.2144e-01, PNorm = 39.3109, GNorm = 3.6048, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.496050
Epoch 85
Validation binary_cross_entropy = 0.522389
Epoch 86
Validation binary_cross_entropy = 0.443462
Epoch 87
Validation binary_cross_entropy = 0.370490
Epoch 88
Validation binary_cross_entropy = 0.358423
Epoch 89
Loss = 4.6016e-01, PNorm = 39.3519, GNorm = 2.7702, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.389961
Epoch 90
Validation binary_cross_entropy = 0.401567
Epoch 91
Validation binary_cross_entropy = 0.397285
Epoch 92
Validation binary_cross_entropy = 0.403945
Epoch 93
Validation binary_cross_entropy = 0.454372
Epoch 94
Loss = 4.2157e-01, PNorm = 39.3956, GNorm = 2.6492, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.503645
Epoch 95
Validation binary_cross_entropy = 0.443793
Epoch 96
Validation binary_cross_entropy = 0.398063
Epoch 97
Validation binary_cross_entropy = 0.431803
Epoch 98
Validation binary_cross_entropy = 0.557907
Epoch 99
Loss = 4.2402e-01, PNorm = 39.4346, GNorm = 2.6328, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.659715
Epoch 100
Validation binary_cross_entropy = 0.722895
Epoch 101
Validation binary_cross_entropy = 0.448481
Epoch 102
Validation binary_cross_entropy = 0.343358
Epoch 103
Validation binary_cross_entropy = 0.373372
Epoch 104
Loss = 4.8415e-01, PNorm = 39.4744, GNorm = 3.1599, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.490418
Epoch 105
Validation binary_cross_entropy = 0.574720
Epoch 106
Validation binary_cross_entropy = 0.498387
Epoch 107
Validation binary_cross_entropy = 0.363763
Epoch 108
Validation binary_cross_entropy = 0.343225
Epoch 109
Loss = 4.6293e-01, PNorm = 39.5183, GNorm = 3.6214, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.363291
Epoch 110
Validation binary_cross_entropy = 0.446586
Epoch 111
Validation binary_cross_entropy = 0.521848
Epoch 112
Validation binary_cross_entropy = 0.499716
Epoch 113
Validation binary_cross_entropy = 0.434555
Epoch 114
Loss = 4.1109e-01, PNorm = 39.5622, GNorm = 2.8289, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.372063
Epoch 115
Validation binary_cross_entropy = 0.356319
Epoch 116
Validation binary_cross_entropy = 0.443627
Epoch 117
Validation binary_cross_entropy = 0.538819
Epoch 118
Validation binary_cross_entropy = 0.523311
Epoch 119
Loss = 4.6394e-01, PNorm = 39.6052, GNorm = 2.9862, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.444505
Epoch 120
Validation binary_cross_entropy = 0.423869
Epoch 121
Validation binary_cross_entropy = 0.398758
Epoch 122
Validation binary_cross_entropy = 0.399687
Epoch 123
Validation binary_cross_entropy = 0.428250
Epoch 124
Loss = 4.4283e-01, PNorm = 39.6456, GNorm = 2.4640, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.443493
Epoch 125
Validation binary_cross_entropy = 0.408050
Epoch 126
Validation binary_cross_entropy = 0.395355
Epoch 127
Validation binary_cross_entropy = 0.473921
Epoch 128
Validation binary_cross_entropy = 0.539669
Epoch 129
Loss = 4.2453e-01, PNorm = 39.6843, GNorm = 2.7567, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.566693
Epoch 130
Validation binary_cross_entropy = 0.506040
Epoch 131
Validation binary_cross_entropy = 0.442820
Epoch 132
Validation binary_cross_entropy = 0.422706
Epoch 133
Validation binary_cross_entropy = 0.410733
Epoch 134
Loss = 4.9128e-01, PNorm = 39.7257, GNorm = 2.9465, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.410225
Epoch 135
Validation binary_cross_entropy = 0.374820
Epoch 136
Validation binary_cross_entropy = 0.371377
Epoch 137
Validation binary_cross_entropy = 0.378497
Epoch 138
Validation binary_cross_entropy = 0.448111
Epoch 139
Loss = 4.4149e-01, PNorm = 39.7635, GNorm = 2.3788, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.477321
Epoch 140
Validation binary_cross_entropy = 0.493482
Epoch 141
Validation binary_cross_entropy = 0.442454
Epoch 142
Validation binary_cross_entropy = 0.389221
Epoch 143
Validation binary_cross_entropy = 0.375784
Epoch 144
Loss = 4.3754e-01, PNorm = 39.8068, GNorm = 2.9729, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.371829
Epoch 145
Validation binary_cross_entropy = 0.384546
Epoch 146
Validation binary_cross_entropy = 0.431895
Epoch 147
Validation binary_cross_entropy = 0.436499
Epoch 148
Validation binary_cross_entropy = 0.410048
Epoch 149
Loss = 3.8374e-01, PNorm = 39.8477, GNorm = 2.6773, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.359065
Epoch 150
Validation binary_cross_entropy = 0.338505
Epoch 151
Validation binary_cross_entropy = 0.349383
Epoch 152
Validation binary_cross_entropy = 0.398157
Epoch 153
Validation binary_cross_entropy = 0.425935
Epoch 154
Loss = 4.5122e-01, PNorm = 39.8861, GNorm = 3.3743, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.417255
Epoch 155
Validation binary_cross_entropy = 0.418789
Epoch 156
Validation binary_cross_entropy = 0.429966
Epoch 157
Validation binary_cross_entropy = 0.416131
Epoch 158
Validation binary_cross_entropy = 0.370090
Epoch 159
Loss = 3.7788e-01, PNorm = 39.9263, GNorm = 2.3391, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.337429
Epoch 160
Validation binary_cross_entropy = 0.345938
Epoch 161
Validation binary_cross_entropy = 0.342412
Epoch 162
Validation binary_cross_entropy = 0.350533
Epoch 163
Validation binary_cross_entropy = 0.400956
Epoch 164
Loss = 3.4319e-01, PNorm = 39.9688, GNorm = 3.3390, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.460916
Epoch 165
Validation binary_cross_entropy = 0.544047
Epoch 166
Validation binary_cross_entropy = 0.498458
Epoch 167
Validation binary_cross_entropy = 0.435205
Epoch 168
Validation binary_cross_entropy = 0.380890
Epoch 169
Loss = 4.4216e-01, PNorm = 40.0108, GNorm = 2.3421, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.364477
Epoch 170
Validation binary_cross_entropy = 0.382122
Epoch 171
Validation binary_cross_entropy = 0.424893
Epoch 172
Validation binary_cross_entropy = 0.449794
Epoch 173
Validation binary_cross_entropy = 0.441283
Epoch 174
Loss = 3.9409e-01, PNorm = 40.0493, GNorm = 2.8460, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.439301
Epoch 175
Validation binary_cross_entropy = 0.411560
Epoch 176
Validation binary_cross_entropy = 0.369708
Epoch 177
Validation binary_cross_entropy = 0.374365
Epoch 178
Validation binary_cross_entropy = 0.394839
Epoch 179
Loss = 4.3690e-01, PNorm = 40.0854, GNorm = 2.1865, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.442184
Epoch 180
Validation binary_cross_entropy = 0.461337
Epoch 181
Validation binary_cross_entropy = 0.416646
Epoch 182
Validation binary_cross_entropy = 0.383356
Epoch 183
Validation binary_cross_entropy = 0.361635
Epoch 184
Loss = 5.0169e-01, PNorm = 40.1275, GNorm = 3.1275, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.346173
Epoch 185
Validation binary_cross_entropy = 0.357167
Epoch 186
Validation binary_cross_entropy = 0.405026
Epoch 187
Validation binary_cross_entropy = 0.467511
Epoch 188
Validation binary_cross_entropy = 0.485793
Epoch 189
Loss = 5.0170e-01, PNorm = 40.1681, GNorm = 4.1333, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.405323
Epoch 190
Validation binary_cross_entropy = 0.354260
Epoch 191
Validation binary_cross_entropy = 0.334633
Epoch 192
Validation binary_cross_entropy = 0.346136
Epoch 193
Validation binary_cross_entropy = 0.391982
Epoch 194
Loss = 4.0490e-01, PNorm = 40.2077, GNorm = 2.0721, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.445490
Epoch 195
Validation binary_cross_entropy = 0.455132
Epoch 196
Validation binary_cross_entropy = 0.401724
Epoch 197
Validation binary_cross_entropy = 0.358265
Epoch 198
Validation binary_cross_entropy = 0.351708
Epoch 199
Loss = 4.0915e-01, PNorm = 40.2468, GNorm = 3.6336, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.374758
Epoch 200
Validation binary_cross_entropy = 0.424888
Epoch 201
Validation binary_cross_entropy = 0.454084
Epoch 202
Validation binary_cross_entropy = 0.454985
Epoch 203
Validation binary_cross_entropy = 0.418351
Epoch 204
Loss = 4.3998e-01, PNorm = 40.2861, GNorm = 2.5464, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.364814
Epoch 205
Validation binary_cross_entropy = 0.350292
Epoch 206
Validation binary_cross_entropy = 0.377649
Epoch 207
Validation binary_cross_entropy = 0.454426
Epoch 208
Validation binary_cross_entropy = 0.526560
Epoch 209
Loss = 3.1746e-01, PNorm = 40.3241, GNorm = 2.8361, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.555911
Epoch 210
Validation binary_cross_entropy = 0.494689
Epoch 211
Validation binary_cross_entropy = 0.419530
Epoch 212
Validation binary_cross_entropy = 0.388485
Epoch 213
Validation binary_cross_entropy = 0.376540
Epoch 214
Loss = 3.3558e-01, PNorm = 40.3647, GNorm = 3.6896, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.382993
Epoch 215
Validation binary_cross_entropy = 0.418819
Epoch 216
Validation binary_cross_entropy = 0.455451
Epoch 217
Validation binary_cross_entropy = 0.462336
Epoch 218
Validation binary_cross_entropy = 0.436587
Epoch 219
Loss = 3.8863e-01, PNorm = 40.4004, GNorm = 3.4572, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.394278
Epoch 220
Validation binary_cross_entropy = 0.344933
Epoch 221
Validation binary_cross_entropy = 0.319581
Epoch 222
Validation binary_cross_entropy = 0.319375
Epoch 223
Validation binary_cross_entropy = 0.354856
Epoch 224
Loss = 3.4845e-01, PNorm = 40.4347, GNorm = 3.3237, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.411780
Epoch 225
Validation binary_cross_entropy = 0.448921
Epoch 226
Validation binary_cross_entropy = 0.464071
Epoch 227
Validation binary_cross_entropy = 0.419610
Epoch 228
Validation binary_cross_entropy = 0.391055
Epoch 229
Loss = 3.1634e-01, PNorm = 40.4684, GNorm = 2.0205, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.370432
Epoch 230
Validation binary_cross_entropy = 0.365043
Epoch 231
Validation binary_cross_entropy = 0.366788
Epoch 232
Validation binary_cross_entropy = 0.388492
Epoch 233
Validation binary_cross_entropy = 0.423277
Epoch 234
Loss = 3.1745e-01, PNorm = 40.5001, GNorm = 2.3571, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.426542
Epoch 235
Validation binary_cross_entropy = 0.408897
Epoch 236
Validation binary_cross_entropy = 0.366842
Epoch 237
Validation binary_cross_entropy = 0.344728
Epoch 238
Validation binary_cross_entropy = 0.346459
Epoch 239
Loss = 4.0021e-01, PNorm = 40.5280, GNorm = 2.4438, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.364754
Epoch 240
Validation binary_cross_entropy = 0.385821
Epoch 241
Validation binary_cross_entropy = 0.404001
Epoch 242
Validation binary_cross_entropy = 0.382796
Epoch 243
Validation binary_cross_entropy = 0.351981
Epoch 244
Loss = 3.7235e-01, PNorm = 40.5561, GNorm = 3.2434, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.338446
Epoch 245
Validation binary_cross_entropy = 0.353413
Epoch 246
Validation binary_cross_entropy = 0.399264
Epoch 247
Validation binary_cross_entropy = 0.442978
Epoch 248
Validation binary_cross_entropy = 0.448353
Epoch 249
Loss = 2.5383e-01, PNorm = 40.5862, GNorm = 2.0321, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.439698
Epoch 250
Validation binary_cross_entropy = 0.414907
Epoch 251
Validation binary_cross_entropy = 0.412066
Epoch 252
Validation binary_cross_entropy = 0.414005
Epoch 253
Validation binary_cross_entropy = 0.420795
Epoch 254
Loss = 3.3114e-01, PNorm = 40.6181, GNorm = 2.5984, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.430532
Epoch 255
Validation binary_cross_entropy = 0.413358
Epoch 256
Validation binary_cross_entropy = 0.392162
Epoch 257
Validation binary_cross_entropy = 0.373022
Epoch 258
Validation binary_cross_entropy = 0.367374
Epoch 259
Loss = 3.1516e-01, PNorm = 40.6481, GNorm = 2.4192, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.369293
Epoch 260
Validation binary_cross_entropy = 0.401676
Epoch 261
Validation binary_cross_entropy = 0.440194
Epoch 262
Validation binary_cross_entropy = 0.475202
Epoch 263
Validation binary_cross_entropy = 0.455661
Epoch 264
Loss = 3.3447e-01, PNorm = 40.6806, GNorm = 2.4859, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.427172
Epoch 265
Validation binary_cross_entropy = 0.398453
Epoch 266
Validation binary_cross_entropy = 0.377391
Epoch 267
Validation binary_cross_entropy = 0.375758
Epoch 268
Validation binary_cross_entropy = 0.386206
Epoch 269
Loss = 2.8560e-01, PNorm = 40.7143, GNorm = 1.9869, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.416642
Epoch 270
Validation binary_cross_entropy = 0.423120
Epoch 271
Validation binary_cross_entropy = 0.400631
Epoch 272
Validation binary_cross_entropy = 0.384030
Epoch 273
Validation binary_cross_entropy = 0.371816
Epoch 274
Loss = 3.0291e-01, PNorm = 40.7457, GNorm = 2.2058, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.378564
Epoch 275
Validation binary_cross_entropy = 0.410989
Epoch 276
Validation binary_cross_entropy = 0.467193
Epoch 277
Validation binary_cross_entropy = 0.461054
Epoch 278
Validation binary_cross_entropy = 0.411478
Epoch 279
Loss = 2.7295e-01, PNorm = 40.7793, GNorm = 2.1865, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.362352
Epoch 280
Validation binary_cross_entropy = 0.346992
Epoch 281
Validation binary_cross_entropy = 0.343334
Epoch 282
Validation binary_cross_entropy = 0.368368
Epoch 283
Validation binary_cross_entropy = 0.402840
Epoch 284
Loss = 3.4801e-01, PNorm = 40.8109, GNorm = 2.7740, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.402251
Epoch 285
Validation binary_cross_entropy = 0.373777
Epoch 286
Validation binary_cross_entropy = 0.348042
Epoch 287
Validation binary_cross_entropy = 0.338914
Epoch 288
Validation binary_cross_entropy = 0.339292
Epoch 289
Loss = 4.5493e-01, PNorm = 40.8380, GNorm = 3.3814, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.345280
Epoch 290
Validation binary_cross_entropy = 0.364964
Epoch 291
Validation binary_cross_entropy = 0.378703
Epoch 292
Validation binary_cross_entropy = 0.394696
Epoch 293
Validation binary_cross_entropy = 0.393100
Epoch 294
Loss = 3.7575e-01, PNorm = 40.8646, GNorm = 2.3261, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.365106
Epoch 295
Validation binary_cross_entropy = 0.333462
Epoch 296
Validation binary_cross_entropy = 0.330230
Epoch 297
Validation binary_cross_entropy = 0.338175
Epoch 298
Validation binary_cross_entropy = 0.376489
Epoch 299
Loss = 3.0323e-01, PNorm = 40.8935, GNorm = 2.1505, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.407690
Model 0 best validation binary_cross_entropy = 0.290353 on epoch 17
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.200195
Ensemble test binary_cross_entropy = 0.200195
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 1,303,001
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.317476
Epoch 1
Validation binary_cross_entropy = 0.456749
Epoch 2
Validation binary_cross_entropy = 0.572252
Epoch 3
Validation binary_cross_entropy = 0.444967
Epoch 4
Loss = 9.7050e-01, PNorm = 38.3078, GNorm = 8.9083, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.407791
Epoch 5
Validation binary_cross_entropy = 0.544707
Epoch 6
Validation binary_cross_entropy = 0.561647
Epoch 7
Validation binary_cross_entropy = 0.396527
Epoch 8
Validation binary_cross_entropy = 0.349855
Epoch 9
Loss = 9.5900e-01, PNorm = 38.3926, GNorm = 8.2744, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.923773
Epoch 10
Validation binary_cross_entropy = 1.257300
Epoch 11
Validation binary_cross_entropy = 0.305556
Epoch 12
Validation binary_cross_entropy = 0.348612
Epoch 13
Validation binary_cross_entropy = 0.435154
Epoch 14
Loss = 9.4972e-01, PNorm = 38.5052, GNorm = 8.3762, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.501097
Epoch 15
Validation binary_cross_entropy = 0.471422
Epoch 16
Validation binary_cross_entropy = 0.384196
Epoch 17
Validation binary_cross_entropy = 0.371699
Epoch 18
Validation binary_cross_entropy = 0.462518
Epoch 19
Loss = 6.3989e-01, PNorm = 38.6113, GNorm = 4.2422, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.607300
Epoch 20
Validation binary_cross_entropy = 0.326907
Epoch 21
Validation binary_cross_entropy = 0.338917
Epoch 22
Validation binary_cross_entropy = 0.391454
Epoch 23
Validation binary_cross_entropy = 0.501769
Epoch 24
Loss = 5.5080e-01, PNorm = 38.7007, GNorm = 4.1739, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.413135
Epoch 25
Validation binary_cross_entropy = 0.343713
Epoch 26
Validation binary_cross_entropy = 0.332034
Epoch 27
Validation binary_cross_entropy = 0.366951
Epoch 28
Validation binary_cross_entropy = 0.480121
Epoch 29
Loss = 7.8876e-01, PNorm = 38.7749, GNorm = 5.7930, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.300528
Epoch 30
Validation binary_cross_entropy = 0.299160
Epoch 31
Validation binary_cross_entropy = 0.364067
Epoch 32
Validation binary_cross_entropy = 0.659520
Epoch 33
Validation binary_cross_entropy = 0.436332
Epoch 34
Loss = 5.7649e-01, PNorm = 38.8390, GNorm = 3.4013, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.337495
Epoch 35
Validation binary_cross_entropy = 0.390899
Epoch 36
Validation binary_cross_entropy = 0.645869
Epoch 37
Validation binary_cross_entropy = 0.604708
Epoch 38
Validation binary_cross_entropy = 0.394246
Epoch 39
Loss = 6.5416e-01, PNorm = 38.8990, GNorm = 4.0352, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.366676
Epoch 40
Validation binary_cross_entropy = 0.386870
Epoch 41
Validation binary_cross_entropy = 0.545711
Epoch 42
Validation binary_cross_entropy = 0.831538
Epoch 43
Validation binary_cross_entropy = 0.544487
Epoch 44
Loss = 5.5699e-01, PNorm = 38.9533, GNorm = 4.1420, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.425787
Epoch 45
Validation binary_cross_entropy = 0.468148
Epoch 46
Validation binary_cross_entropy = 0.663730
Epoch 47
Validation binary_cross_entropy = 0.629526
Epoch 48
Validation binary_cross_entropy = 0.432314
Epoch 49
Loss = 6.7848e-01, PNorm = 39.0010, GNorm = 3.9038, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.378171
Epoch 50
Validation binary_cross_entropy = 0.392001
Epoch 51
Validation binary_cross_entropy = 0.512287
Epoch 52
Validation binary_cross_entropy = 0.641331
Epoch 53
Validation binary_cross_entropy = 0.646116
Epoch 54
Loss = 4.9001e-01, PNorm = 39.0455, GNorm = 3.1644, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.502729
Epoch 55
Validation binary_cross_entropy = 0.462043
Epoch 56
Validation binary_cross_entropy = 0.499330
Epoch 57
Validation binary_cross_entropy = 0.576037
Epoch 58
Validation binary_cross_entropy = 0.572802
Epoch 59
Loss = 5.9896e-01, PNorm = 39.0894, GNorm = 4.8392, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.527211
Epoch 60
Validation binary_cross_entropy = 0.455381
Epoch 61
Validation binary_cross_entropy = 0.422659
Epoch 62
Validation binary_cross_entropy = 0.445575
Epoch 63
Validation binary_cross_entropy = 0.539283
Epoch 64
Loss = 6.0548e-01, PNorm = 39.1326, GNorm = 3.0017, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.638785
Epoch 65
Validation binary_cross_entropy = 0.593496
Epoch 66
Validation binary_cross_entropy = 0.504403
Epoch 67
Validation binary_cross_entropy = 0.425571
Epoch 68
Validation binary_cross_entropy = 0.429602
Epoch 69
Loss = 4.4355e-01, PNorm = 39.1758, GNorm = 2.7587, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.524971
Epoch 70
Validation binary_cross_entropy = 0.644170
Epoch 71
Validation binary_cross_entropy = 0.525185
Epoch 72
Validation binary_cross_entropy = 0.425468
Epoch 73
Validation binary_cross_entropy = 0.428555
Epoch 74
Loss = 4.6980e-01, PNorm = 39.2181, GNorm = 3.3013, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.563678
Epoch 75
Validation binary_cross_entropy = 0.689607
Epoch 76
Validation binary_cross_entropy = 0.603000
Epoch 77
Validation binary_cross_entropy = 0.480710
Epoch 78
Validation binary_cross_entropy = 0.479114
Epoch 79
Loss = 5.4061e-01, PNorm = 39.2608, GNorm = 2.5438, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.531910
Epoch 80
Validation binary_cross_entropy = 0.611760
Epoch 81
Validation binary_cross_entropy = 0.616208
Epoch 82
Validation binary_cross_entropy = 0.549798
Epoch 83
Validation binary_cross_entropy = 0.465543
Epoch 84
Loss = 5.2674e-01, PNorm = 39.3021, GNorm = 2.7733, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.453614
Epoch 85
Validation binary_cross_entropy = 0.489832
Epoch 86
Validation binary_cross_entropy = 0.571000
Epoch 87
Validation binary_cross_entropy = 0.583408
Epoch 88
Validation binary_cross_entropy = 0.537326
Epoch 89
Loss = 5.3909e-01, PNorm = 39.3466, GNorm = 3.8314, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.457394
Epoch 90
Validation binary_cross_entropy = 0.434392
Epoch 91
Validation binary_cross_entropy = 0.453870
Epoch 92
Validation binary_cross_entropy = 0.580695
Epoch 93
Validation binary_cross_entropy = 0.617840
Epoch 94
Loss = 4.6687e-01, PNorm = 39.3914, GNorm = 2.9780, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.491049
Epoch 95
Validation binary_cross_entropy = 0.424684
Epoch 96
Validation binary_cross_entropy = 0.425870
Epoch 97
Validation binary_cross_entropy = 0.479616
Epoch 98
Validation binary_cross_entropy = 0.571658
Epoch 99
Loss = 4.5259e-01, PNorm = 39.4348, GNorm = 3.0876, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.477432
Epoch 100
Validation binary_cross_entropy = 0.421569
Epoch 101
Validation binary_cross_entropy = 0.365618
Epoch 102
Validation binary_cross_entropy = 0.363543
Epoch 103
Validation binary_cross_entropy = 0.400296
Epoch 104
Loss = 5.7211e-01, PNorm = 39.4775, GNorm = 2.9221, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.465742
Epoch 105
Validation binary_cross_entropy = 0.498578
Epoch 106
Validation binary_cross_entropy = 0.467059
Epoch 107
Validation binary_cross_entropy = 0.403169
Epoch 108
Validation binary_cross_entropy = 0.369281
Epoch 109
Loss = 4.5388e-01, PNorm = 39.5192, GNorm = 3.4178, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.371267
Epoch 110
Validation binary_cross_entropy = 0.420464
Epoch 111
Validation binary_cross_entropy = 0.453303
Epoch 112
Validation binary_cross_entropy = 0.453051
Epoch 113
Validation binary_cross_entropy = 0.403131
Epoch 114
Loss = 4.8134e-01, PNorm = 39.5601, GNorm = 2.4119, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.375964
Epoch 115
Validation binary_cross_entropy = 0.371659
Epoch 116
Validation binary_cross_entropy = 0.395231
Epoch 117
Validation binary_cross_entropy = 0.413797
Epoch 118
Validation binary_cross_entropy = 0.409098
Epoch 119
Loss = 3.8450e-01, PNorm = 39.5977, GNorm = 2.4974, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.427378
Epoch 120
Validation binary_cross_entropy = 0.400852
Epoch 121
Validation binary_cross_entropy = 0.357314
Epoch 122
Validation binary_cross_entropy = 0.360827
Epoch 123
Validation binary_cross_entropy = 0.390821
Epoch 124
Loss = 3.9344e-01, PNorm = 39.6358, GNorm = 2.1130, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.470920
Epoch 125
Validation binary_cross_entropy = 0.585911
Epoch 126
Validation binary_cross_entropy = 0.579235
Epoch 127
Validation binary_cross_entropy = 0.530256
Epoch 128
Validation binary_cross_entropy = 0.444578
Epoch 129
Loss = 4.3731e-01, PNorm = 39.6768, GNorm = 4.1307, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.392821
Epoch 130
Validation binary_cross_entropy = 0.425936
Epoch 131
Validation binary_cross_entropy = 0.439607
Epoch 132
Validation binary_cross_entropy = 0.402751
Epoch 133
Validation binary_cross_entropy = 0.423669
Epoch 134
Loss = 3.3477e-01, PNorm = 39.7207, GNorm = 2.2619, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.463012
Epoch 135
Validation binary_cross_entropy = 0.455732
Epoch 136
Validation binary_cross_entropy = 0.435640
Epoch 137
Validation binary_cross_entropy = 0.422526
Epoch 138
Validation binary_cross_entropy = 0.385159
Epoch 139
Loss = 4.6014e-01, PNorm = 39.7602, GNorm = 3.0557, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.371890
Epoch 140
Validation binary_cross_entropy = 0.457456
Epoch 141
Validation binary_cross_entropy = 0.635471
Epoch 142
Validation binary_cross_entropy = 0.608970
Epoch 143
Validation binary_cross_entropy = 0.450794
Epoch 144
Loss = 4.9487e-01, PNorm = 39.8003, GNorm = 3.3972, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.342391
Epoch 145
Validation binary_cross_entropy = 0.338920
Epoch 146
Validation binary_cross_entropy = 0.386581
Epoch 147
Validation binary_cross_entropy = 0.438780
Epoch 148
Validation binary_cross_entropy = 0.468304
Epoch 149
Loss = 4.3857e-01, PNorm = 39.8416, GNorm = 2.3189, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.477056
Epoch 150
Validation binary_cross_entropy = 0.441949
Epoch 151
Validation binary_cross_entropy = 0.407065
Epoch 152
Validation binary_cross_entropy = 0.400301
Epoch 153
Validation binary_cross_entropy = 0.403703
Epoch 154
Loss = 5.0844e-01, PNorm = 39.8830, GNorm = 3.5377, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.421924
Epoch 155
Validation binary_cross_entropy = 0.479570
Epoch 156
Validation binary_cross_entropy = 0.566367
Epoch 157
Validation binary_cross_entropy = 0.667078
Epoch 158
Validation binary_cross_entropy = 0.612759
Epoch 159
Loss = 4.7149e-01, PNorm = 39.9216, GNorm = 3.0290, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.398933
Epoch 160
Validation binary_cross_entropy = 0.342474
Epoch 161
Validation binary_cross_entropy = 0.344697
Epoch 162
Validation binary_cross_entropy = 0.413120
Epoch 163
Validation binary_cross_entropy = 0.482300
Epoch 164
Loss = 3.7760e-01, PNorm = 39.9598, GNorm = 2.2644, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.498258
Epoch 165
Validation binary_cross_entropy = 0.485791
Epoch 166
Validation binary_cross_entropy = 0.402540
Epoch 167
Validation binary_cross_entropy = 0.359132
Epoch 168
Validation binary_cross_entropy = 0.364298
Epoch 169
Loss = 4.3254e-01, PNorm = 39.9976, GNorm = 2.6201, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.402584
Epoch 170
Validation binary_cross_entropy = 0.417791
Epoch 171
Validation binary_cross_entropy = 0.424852
Epoch 172
Validation binary_cross_entropy = 0.431909
Epoch 173
Validation binary_cross_entropy = 0.417570
Epoch 174
Loss = 3.9249e-01, PNorm = 40.0356, GNorm = 2.2873, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.418503
Epoch 175
Validation binary_cross_entropy = 0.423102
Epoch 176
Validation binary_cross_entropy = 0.413143
Epoch 177
Validation binary_cross_entropy = 0.437590
Epoch 178
Validation binary_cross_entropy = 0.509749
Epoch 179
Loss = 4.0949e-01, PNorm = 40.0760, GNorm = 2.4379, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.556275
Epoch 180
Validation binary_cross_entropy = 0.528578
Epoch 181
Validation binary_cross_entropy = 0.472675
Epoch 182
Validation binary_cross_entropy = 0.414562
Epoch 183
Validation binary_cross_entropy = 0.375687
Epoch 184
Loss = 4.2706e-01, PNorm = 40.1182, GNorm = 2.8345, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.359585
Epoch 185
Validation binary_cross_entropy = 0.348214
Epoch 186
Validation binary_cross_entropy = 0.361376
Epoch 187
Validation binary_cross_entropy = 0.388106
Epoch 188
Validation binary_cross_entropy = 0.425595
Epoch 189
Loss = 3.5127e-01, PNorm = 40.1553, GNorm = 2.7115, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.399477
Epoch 190
Validation binary_cross_entropy = 0.392046
Epoch 191
Validation binary_cross_entropy = 0.403375
Epoch 192
Validation binary_cross_entropy = 0.434999
Epoch 193
Validation binary_cross_entropy = 0.452141
Epoch 194
Loss = 4.4641e-01, PNorm = 40.1940, GNorm = 3.0519, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.474729
Epoch 195
Validation binary_cross_entropy = 0.448967
Epoch 196
Validation binary_cross_entropy = 0.405490
Epoch 197
Validation binary_cross_entropy = 0.385811
Epoch 198
Validation binary_cross_entropy = 0.376311
Epoch 199
Loss = 4.5579e-01, PNorm = 40.2329, GNorm = 2.5245, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.388001
Epoch 200
Validation binary_cross_entropy = 0.397847
Epoch 201
Validation binary_cross_entropy = 0.417679
Epoch 202
Validation binary_cross_entropy = 0.463231
Epoch 203
Validation binary_cross_entropy = 0.479063
Epoch 204
Loss = 3.3861e-01, PNorm = 40.2714, GNorm = 2.9964, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.455045
Epoch 205
Validation binary_cross_entropy = 0.433399
Epoch 206
Validation binary_cross_entropy = 0.383846
Epoch 207
Validation binary_cross_entropy = 0.343571
Epoch 208
Validation binary_cross_entropy = 0.333425
Epoch 209
Loss = 4.3182e-01, PNorm = 40.3076, GNorm = 3.1808, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.376765
Epoch 210
Validation binary_cross_entropy = 0.445996
Epoch 211
Validation binary_cross_entropy = 0.466789
Epoch 212
Validation binary_cross_entropy = 0.464444
Epoch 213
Validation binary_cross_entropy = 0.437454
Epoch 214
Loss = 4.2296e-01, PNorm = 40.3438, GNorm = 2.9464, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.394628
Epoch 215
Validation binary_cross_entropy = 0.381189
Epoch 216
Validation binary_cross_entropy = 0.381709
Epoch 217
Validation binary_cross_entropy = 0.403434
Epoch 218
Validation binary_cross_entropy = 0.430790
Epoch 219
Loss = 2.6007e-01, PNorm = 40.3811, GNorm = 1.8174, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.451060
Epoch 220
Validation binary_cross_entropy = 0.447761
Epoch 221
Validation binary_cross_entropy = 0.413327
Epoch 222
Validation binary_cross_entropy = 0.398952
Epoch 223
Validation binary_cross_entropy = 0.385796
Epoch 224
Loss = 3.1545e-01, PNorm = 40.4155, GNorm = 2.1456, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.376736
Epoch 225
Validation binary_cross_entropy = 0.368533
Epoch 226
Validation binary_cross_entropy = 0.392422
Epoch 227
Validation binary_cross_entropy = 0.427169
Epoch 228
Validation binary_cross_entropy = 0.462795
Epoch 229
Loss = 2.9445e-01, PNorm = 40.4470, GNorm = 2.3253, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.463921
Epoch 230
Validation binary_cross_entropy = 0.417671
Epoch 231
Validation binary_cross_entropy = 0.360645
Epoch 232
Validation binary_cross_entropy = 0.379304
Epoch 233
Validation binary_cross_entropy = 0.442902
Epoch 234
Loss = 2.7644e-01, PNorm = 40.4810, GNorm = 2.9510, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.495292
Epoch 235
Validation binary_cross_entropy = 0.435307
Epoch 236
Validation binary_cross_entropy = 0.371752
Epoch 237
Validation binary_cross_entropy = 0.343624
Epoch 238
Validation binary_cross_entropy = 0.341534
Epoch 239
Loss = 3.4356e-01, PNorm = 40.5141, GNorm = 2.8417, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.350686
Epoch 240
Validation binary_cross_entropy = 0.398573
Epoch 241
Validation binary_cross_entropy = 0.447882
Epoch 242
Validation binary_cross_entropy = 0.470892
Epoch 243
Validation binary_cross_entropy = 0.446582
Epoch 244
Loss = 3.3580e-01, PNorm = 40.5463, GNorm = 1.8856, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.406746
Epoch 245
Validation binary_cross_entropy = 0.383508
Epoch 246
Validation binary_cross_entropy = 0.380718
Epoch 247
Validation binary_cross_entropy = 0.394688
Epoch 248
Validation binary_cross_entropy = 0.415885
Epoch 249
Loss = 3.3694e-01, PNorm = 40.5793, GNorm = 2.7540, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.451091
Epoch 250
Validation binary_cross_entropy = 0.453931
Epoch 251
Validation binary_cross_entropy = 0.416899
Epoch 252
Validation binary_cross_entropy = 0.387452
Epoch 253
Validation binary_cross_entropy = 0.388902
Epoch 254
Loss = 3.0771e-01, PNorm = 40.6124, GNorm = 2.2829, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.422467
Epoch 255
Validation binary_cross_entropy = 0.431147
Epoch 256
Validation binary_cross_entropy = 0.435035
Epoch 257
Validation binary_cross_entropy = 0.438595
Epoch 258
Validation binary_cross_entropy = 0.436012
Epoch 259
Loss = 2.8429e-01, PNorm = 40.6443, GNorm = 1.7721, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.426202
Epoch 260
Validation binary_cross_entropy = 0.423408
Epoch 261
Validation binary_cross_entropy = 0.424678
Epoch 262
Validation binary_cross_entropy = 0.412942
Epoch 263
Validation binary_cross_entropy = 0.402112
Epoch 264
Loss = 3.0196e-01, PNorm = 40.6773, GNorm = 2.2233, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.392911
Epoch 265
Validation binary_cross_entropy = 0.411917
Epoch 266
Validation binary_cross_entropy = 0.438018
Epoch 267
Validation binary_cross_entropy = 0.446272
Epoch 268
Validation binary_cross_entropy = 0.438358
Epoch 269
Loss = 3.9687e-01, PNorm = 40.7081, GNorm = 2.4800, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.427020
Epoch 270
Validation binary_cross_entropy = 0.400471
Epoch 271
Validation binary_cross_entropy = 0.383598
Epoch 272
Validation binary_cross_entropy = 0.385550
Epoch 273
Validation binary_cross_entropy = 0.407962
Epoch 274
Loss = 2.7659e-01, PNorm = 40.7378, GNorm = 2.3695, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.462672
Epoch 275
Validation binary_cross_entropy = 0.478408
Epoch 276
Validation binary_cross_entropy = 0.464777
Epoch 277
Validation binary_cross_entropy = 0.416769
Epoch 278
Validation binary_cross_entropy = 0.374168
Epoch 279
Loss = 3.0112e-01, PNorm = 40.7751, GNorm = 2.5636, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.359655
Epoch 280
Validation binary_cross_entropy = 0.370354
Epoch 281
Validation binary_cross_entropy = 0.393414
Epoch 282
Validation binary_cross_entropy = 0.408560
Epoch 283
Validation binary_cross_entropy = 0.416573
Epoch 284
Loss = 3.0351e-01, PNorm = 40.8096, GNorm = 2.3679, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.408879
Epoch 285
Validation binary_cross_entropy = 0.382023
Epoch 286
Validation binary_cross_entropy = 0.361621
Epoch 287
Validation binary_cross_entropy = 0.369423
Epoch 288
Validation binary_cross_entropy = 0.403756
Epoch 289
Loss = 3.3682e-01, PNorm = 40.8398, GNorm = 2.5179, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.443514
Epoch 290
Validation binary_cross_entropy = 0.473197
Epoch 291
Validation binary_cross_entropy = 0.486495
Epoch 292
Validation binary_cross_entropy = 0.462124
Epoch 293
Validation binary_cross_entropy = 0.418949
Epoch 294
Loss = 3.1721e-01, PNorm = 40.8723, GNorm = 2.0824, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.375868
Epoch 295
Validation binary_cross_entropy = 0.355144
Epoch 296
Validation binary_cross_entropy = 0.348329
Epoch 297
Validation binary_cross_entropy = 0.360205
Epoch 298
Validation binary_cross_entropy = 0.380751
Epoch 299
Loss = 2.8651e-01, PNorm = 40.9026, GNorm = 2.3107, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.401946
Model 0 best validation binary_cross_entropy = 0.299160 on epoch 30
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.207248
Ensemble test binary_cross_entropy = 0.207248
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 1,303,001
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.357418
Epoch 1
Validation binary_cross_entropy = 0.579718
Epoch 2
Validation binary_cross_entropy = 0.476368
Epoch 3
Validation binary_cross_entropy = 0.389625
Epoch 4
Loss = 9.4244e-01, PNorm = 38.3076, GNorm = 8.2429, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.633910
Epoch 5
Validation binary_cross_entropy = 0.688221
Epoch 6
Validation binary_cross_entropy = 0.341898
Epoch 7
Validation binary_cross_entropy = 1.012300
Epoch 8
Validation binary_cross_entropy = 0.430052
Epoch 9
Loss = 9.4409e-01, PNorm = 38.3956, GNorm = 11.3220, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.332945
Epoch 10
Validation binary_cross_entropy = 0.856204
Epoch 11
Validation binary_cross_entropy = 0.686451
Epoch 12
Validation binary_cross_entropy = 0.341579
Epoch 13
Validation binary_cross_entropy = 0.412420
Epoch 14
Loss = 8.7765e-01, PNorm = 38.5106, GNorm = 5.6095, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.634819
Epoch 15
Validation binary_cross_entropy = 0.734383
Epoch 16
Validation binary_cross_entropy = 0.849899
Epoch 17
Validation binary_cross_entropy = 0.312379
Epoch 18
Validation binary_cross_entropy = 0.384304
Epoch 19
Loss = 7.8601e-01, PNorm = 38.6179, GNorm = 7.4339, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.868487
Epoch 20
Validation binary_cross_entropy = 0.415651
Epoch 21
Validation binary_cross_entropy = 0.308237
Epoch 22
Validation binary_cross_entropy = 0.441863
Epoch 23
Validation binary_cross_entropy = 0.474883
Epoch 24
Loss = 5.6732e-01, PNorm = 38.7057, GNorm = 3.5531, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.360440
Epoch 25
Validation binary_cross_entropy = 0.354367
Epoch 26
Validation binary_cross_entropy = 0.425787
Epoch 27
Validation binary_cross_entropy = 0.548110
Epoch 28
Validation binary_cross_entropy = 0.530110
Epoch 29
Loss = 5.6427e-01, PNorm = 38.7750, GNorm = 5.6345, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.418027
Epoch 30
Validation binary_cross_entropy = 0.356948
Epoch 31
Validation binary_cross_entropy = 0.439635
Epoch 32
Validation binary_cross_entropy = 0.781271
Epoch 33
Validation binary_cross_entropy = 0.707778
Epoch 34
Loss = 5.9970e-01, PNorm = 38.8341, GNorm = 5.0645, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.755902
Epoch 35
Validation binary_cross_entropy = 1.037825
Epoch 36
Validation binary_cross_entropy = 1.015862
Epoch 37
Validation binary_cross_entropy = 0.483724
Epoch 38
Validation binary_cross_entropy = 0.322773
Epoch 39
Loss = 7.2086e-01, PNorm = 38.8864, GNorm = 4.6837, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.388187
Epoch 40
Validation binary_cross_entropy = 0.581932
Epoch 41
Validation binary_cross_entropy = 0.620399
Epoch 42
Validation binary_cross_entropy = 0.460858
Epoch 43
Validation binary_cross_entropy = 0.426991
Epoch 44
Loss = 5.4592e-01, PNorm = 38.9370, GNorm = 3.4059, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.527376
Epoch 45
Validation binary_cross_entropy = 0.848715
Epoch 46
Validation binary_cross_entropy = 1.215590
Epoch 47
Validation binary_cross_entropy = 1.223600
Epoch 48
Validation binary_cross_entropy = 1.023851
Epoch 49
Loss = 6.5251e-01, PNorm = 38.9863, GNorm = 4.7318, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.765085
Epoch 50
Validation binary_cross_entropy = 0.569008
Epoch 51
Validation binary_cross_entropy = 0.466293
Epoch 52
Validation binary_cross_entropy = 0.523081
Epoch 53
Validation binary_cross_entropy = 0.559000
Epoch 54
Loss = 4.6826e-01, PNorm = 39.0323, GNorm = 3.1293, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.593443
Epoch 55
Validation binary_cross_entropy = 0.506945
Epoch 56
Validation binary_cross_entropy = 0.455518
Epoch 57
Validation binary_cross_entropy = 0.471171
Epoch 58
Validation binary_cross_entropy = 0.548038
Epoch 59
Loss = 4.6022e-01, PNorm = 39.0712, GNorm = 4.5150, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.691966
Epoch 60
Validation binary_cross_entropy = 0.785536
Epoch 61
Validation binary_cross_entropy = 0.657364
Epoch 62
Validation binary_cross_entropy = 0.578084
Epoch 63
Validation binary_cross_entropy = 0.509126
Epoch 64
Loss = 5.0326e-01, PNorm = 39.1111, GNorm = 3.7184, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.577424
Epoch 65
Validation binary_cross_entropy = 0.654670
Epoch 66
Validation binary_cross_entropy = 0.545065
Epoch 67
Validation binary_cross_entropy = 0.513841
Epoch 68
Validation binary_cross_entropy = 0.529938
Epoch 69
Loss = 4.6781e-01, PNorm = 39.1527, GNorm = 2.5938, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.615793
Epoch 70
Validation binary_cross_entropy = 0.647535
Epoch 71
Validation binary_cross_entropy = 0.579018
Epoch 72
Validation binary_cross_entropy = 0.580717
Epoch 73
Validation binary_cross_entropy = 0.688652
Epoch 74
Loss = 4.6316e-01, PNorm = 39.1973, GNorm = 2.7012, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.752007
Epoch 75
Validation binary_cross_entropy = 0.677216
Epoch 76
Validation binary_cross_entropy = 0.564202
Epoch 77
Validation binary_cross_entropy = 0.490248
Epoch 78
Validation binary_cross_entropy = 0.543789
Epoch 79
Loss = 3.4021e-01, PNorm = 39.2405, GNorm = 2.2700, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.694776
Epoch 80
Validation binary_cross_entropy = 0.785684
Epoch 81
Validation binary_cross_entropy = 0.604403
Epoch 82
Validation binary_cross_entropy = 0.403350
Epoch 83
Validation binary_cross_entropy = 0.422880
Epoch 84
Loss = 4.2890e-01, PNorm = 39.2816, GNorm = 4.9698, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.572878
Epoch 85
Validation binary_cross_entropy = 0.692890
Epoch 86
Validation binary_cross_entropy = 0.502255
Epoch 87
Validation binary_cross_entropy = 0.419839
Epoch 88
Validation binary_cross_entropy = 0.441142
Epoch 89
Loss = 6.3132e-01, PNorm = 39.3243, GNorm = 4.0817, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.481544
Epoch 90
Validation binary_cross_entropy = 0.564304
Epoch 91
Validation binary_cross_entropy = 0.650065
Epoch 92
Validation binary_cross_entropy = 0.618727
Epoch 93
Validation binary_cross_entropy = 0.567021
Epoch 94
Loss = 4.3015e-01, PNorm = 39.3678, GNorm = 3.2569, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.560515
Epoch 95
Validation binary_cross_entropy = 0.614704
Epoch 96
Validation binary_cross_entropy = 0.773405
Epoch 97
Validation binary_cross_entropy = 0.787618
Epoch 98
Validation binary_cross_entropy = 0.659112
Epoch 99
Loss = 4.4932e-01, PNorm = 39.4088, GNorm = 2.7321, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.439946
Epoch 100
Validation binary_cross_entropy = 0.420073
Epoch 101
Validation binary_cross_entropy = 0.451944
Epoch 102
Validation binary_cross_entropy = 0.471190
Epoch 103
Validation binary_cross_entropy = 0.496376
Epoch 104
Loss = 4.8234e-01, PNorm = 39.4500, GNorm = 2.9708, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.547031
Epoch 105
Validation binary_cross_entropy = 0.700318
Epoch 106
Validation binary_cross_entropy = 0.758510
Epoch 107
Validation binary_cross_entropy = 0.672603
Epoch 108
Validation binary_cross_entropy = 0.724487
Epoch 109
Loss = 5.4789e-01, PNorm = 39.4911, GNorm = 3.6704, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.487001
Epoch 110
Validation binary_cross_entropy = 0.419820
Epoch 111
Validation binary_cross_entropy = 0.458214
Epoch 112
Validation binary_cross_entropy = 0.456218
Epoch 113
Validation binary_cross_entropy = 0.467912
Epoch 114
Loss = 3.9543e-01, PNorm = 39.5376, GNorm = 2.2375, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.463996
Epoch 115
Validation binary_cross_entropy = 0.431896
Epoch 116
Validation binary_cross_entropy = 0.415729
Epoch 117
Validation binary_cross_entropy = 0.408815
Epoch 118
Validation binary_cross_entropy = 0.440395
Epoch 119
Loss = 3.5565e-01, PNorm = 39.5822, GNorm = 2.5475, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.467130
Epoch 120
Validation binary_cross_entropy = 0.500779
Epoch 121
Validation binary_cross_entropy = 0.469605
Epoch 122
Validation binary_cross_entropy = 0.426691
Epoch 123
Validation binary_cross_entropy = 0.433759
Epoch 124
Loss = 4.4469e-01, PNorm = 39.6275, GNorm = 3.1852, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.424761
Epoch 125
Validation binary_cross_entropy = 0.484054
Epoch 126
Validation binary_cross_entropy = 0.452022
Epoch 127
Validation binary_cross_entropy = 0.440859
Epoch 128
Validation binary_cross_entropy = 0.430892
Epoch 129
Loss = 5.0709e-01, PNorm = 39.6704, GNorm = 4.2263, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.399539
Epoch 130
Validation binary_cross_entropy = 0.382877
Epoch 131
Validation binary_cross_entropy = 0.414373
Epoch 132
Validation binary_cross_entropy = 0.409520
Epoch 133
Validation binary_cross_entropy = 0.442563
Epoch 134
Loss = 4.6168e-01, PNorm = 39.7086, GNorm = 2.7145, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.472580
Epoch 135
Validation binary_cross_entropy = 0.497549
Epoch 136
Validation binary_cross_entropy = 0.587727
Epoch 137
Validation binary_cross_entropy = 0.572846
Epoch 138
Validation binary_cross_entropy = 0.540879
Epoch 139
Loss = 3.9817e-01, PNorm = 39.7498, GNorm = 2.5574, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.575584
Epoch 140
Validation binary_cross_entropy = 0.620591
Epoch 141
Validation binary_cross_entropy = 0.677304
Epoch 142
Validation binary_cross_entropy = 0.629027
Epoch 143
Validation binary_cross_entropy = 0.607922
Epoch 144
Loss = 5.0272e-01, PNorm = 39.7901, GNorm = 2.8076, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.426086
Epoch 145
Validation binary_cross_entropy = 0.365878
Epoch 146
Validation binary_cross_entropy = 0.395728
Epoch 147
Validation binary_cross_entropy = 0.474437
Epoch 148
Validation binary_cross_entropy = 0.534849
Epoch 149
Loss = 5.7123e-01, PNorm = 39.8305, GNorm = 3.8602, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.601587
Epoch 150
Validation binary_cross_entropy = 0.656841
Epoch 151
Validation binary_cross_entropy = 0.557825
Epoch 152
Validation binary_cross_entropy = 0.557227
Epoch 153
Validation binary_cross_entropy = 0.666817
Epoch 154
Loss = 3.9159e-01, PNorm = 39.8732, GNorm = 2.9470, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.690880
Epoch 155
Validation binary_cross_entropy = 0.609094
Epoch 156
Validation binary_cross_entropy = 0.584631
Epoch 157
Validation binary_cross_entropy = 0.524311
Epoch 158
Validation binary_cross_entropy = 0.492336
Epoch 159
Loss = 4.3020e-01, PNorm = 39.9170, GNorm = 3.1042, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.433010
Epoch 160
Validation binary_cross_entropy = 0.397634
Epoch 161
Validation binary_cross_entropy = 0.418745
Epoch 162
Validation binary_cross_entropy = 0.471506
Epoch 163
Validation binary_cross_entropy = 0.505586
Epoch 164
Loss = 2.9543e-01, PNorm = 39.9561, GNorm = 1.9286, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.502779
Epoch 165
Validation binary_cross_entropy = 0.485381
Epoch 166
Validation binary_cross_entropy = 0.478910
Epoch 167
Validation binary_cross_entropy = 0.472064
Epoch 168
Validation binary_cross_entropy = 0.522255
Epoch 169
Loss = 3.5709e-01, PNorm = 39.9929, GNorm = 2.8349, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.670776
Epoch 170
Validation binary_cross_entropy = 0.772114
Epoch 171
Validation binary_cross_entropy = 0.738949
Epoch 172
Validation binary_cross_entropy = 0.593229
Epoch 173
Validation binary_cross_entropy = 0.573137
Epoch 174
Loss = 3.4441e-01, PNorm = 40.0282, GNorm = 2.8838, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.537252
Epoch 175
Validation binary_cross_entropy = 0.521304
Epoch 176
Validation binary_cross_entropy = 0.534607
Epoch 177
Validation binary_cross_entropy = 0.500267
Epoch 178
Validation binary_cross_entropy = 0.435912
Epoch 179
Loss = 3.9615e-01, PNorm = 40.0665, GNorm = 3.3759, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.446486
Epoch 180
Validation binary_cross_entropy = 0.551618
Epoch 181
Validation binary_cross_entropy = 0.604086
Epoch 182
Validation binary_cross_entropy = 0.487718
Epoch 183
Validation binary_cross_entropy = 0.455502
Epoch 184
Loss = 3.8455e-01, PNorm = 40.1065, GNorm = 3.7780, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.466763
Epoch 185
Validation binary_cross_entropy = 0.521308
Epoch 186
Validation binary_cross_entropy = 0.563024
Epoch 187
Validation binary_cross_entropy = 0.556845
Epoch 188
Validation binary_cross_entropy = 0.494075
Epoch 189
Loss = 3.8904e-01, PNorm = 40.1438, GNorm = 3.3010, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.497858
Epoch 190
Validation binary_cross_entropy = 0.542758
Epoch 191
Validation binary_cross_entropy = 0.533715
Epoch 192
Validation binary_cross_entropy = 0.482693
Epoch 193
Validation binary_cross_entropy = 0.433772
Epoch 194
Loss = 3.3117e-01, PNorm = 40.1786, GNorm = 2.6870, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.412821
Epoch 195
Validation binary_cross_entropy = 0.404611
Epoch 196
Validation binary_cross_entropy = 0.424582
Epoch 197
Validation binary_cross_entropy = 0.451739
Epoch 198
Validation binary_cross_entropy = 0.461057
Epoch 199
Loss = 4.0889e-01, PNorm = 40.2154, GNorm = 2.8161, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.451857
Epoch 200
Validation binary_cross_entropy = 0.444130
Epoch 201
Validation binary_cross_entropy = 0.445979
Epoch 202
Validation binary_cross_entropy = 0.533315
Epoch 203
Validation binary_cross_entropy = 0.561430
Epoch 204
Loss = 2.7174e-01, PNorm = 40.2535, GNorm = 2.3233, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.553014
Epoch 205
Validation binary_cross_entropy = 0.448701
Epoch 206
Validation binary_cross_entropy = 0.394589
Epoch 207
Validation binary_cross_entropy = 0.413981
Epoch 208
Validation binary_cross_entropy = 0.490055
Epoch 209
Loss = 4.0195e-01, PNorm = 40.2940, GNorm = 2.7750, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.491004
Epoch 210
Validation binary_cross_entropy = 0.468148
Epoch 211
Validation binary_cross_entropy = 0.418051
Epoch 212
Validation binary_cross_entropy = 0.413850
Epoch 213
Validation binary_cross_entropy = 0.390390
Epoch 214
Loss = 3.7008e-01, PNorm = 40.3312, GNorm = 3.0584, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.398393
Epoch 215
Validation binary_cross_entropy = 0.382141
Epoch 216
Validation binary_cross_entropy = 0.406416
Epoch 217
Validation binary_cross_entropy = 0.427030
Epoch 218
Validation binary_cross_entropy = 0.464365
Epoch 219
Loss = 2.5430e-01, PNorm = 40.3670, GNorm = 2.0116, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.490640
Epoch 220
Validation binary_cross_entropy = 0.516222
Epoch 221
Validation binary_cross_entropy = 0.540205
Epoch 222
Validation binary_cross_entropy = 0.556868
Epoch 223
Validation binary_cross_entropy = 0.528383
Epoch 224
Loss = 2.4322e-01, PNorm = 40.4019, GNorm = 1.7765, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.476201
Epoch 225
Validation binary_cross_entropy = 0.448245
Epoch 226
Validation binary_cross_entropy = 0.447875
Epoch 227
Validation binary_cross_entropy = 0.461280
Epoch 228
Validation binary_cross_entropy = 0.460547
Epoch 229
Loss = 3.8554e-01, PNorm = 40.4352, GNorm = 2.5318, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.474588
Epoch 230
Validation binary_cross_entropy = 0.439877
Epoch 231
Validation binary_cross_entropy = 0.422769
Epoch 232
Validation binary_cross_entropy = 0.442758
Epoch 233
Validation binary_cross_entropy = 0.441216
Epoch 234
Loss = 3.0875e-01, PNorm = 40.4677, GNorm = 2.1926, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.453519
Epoch 235
Validation binary_cross_entropy = 0.421516
Epoch 236
Validation binary_cross_entropy = 0.361382
Epoch 237
Validation binary_cross_entropy = 0.328839
Epoch 238
Validation binary_cross_entropy = 0.336704
Epoch 239
Loss = 3.4947e-01, PNorm = 40.4981, GNorm = 2.1534, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.369469
Epoch 240
Validation binary_cross_entropy = 0.401056
Epoch 241
Validation binary_cross_entropy = 0.430874
Epoch 242
Validation binary_cross_entropy = 0.440130
Epoch 243
Validation binary_cross_entropy = 0.458092
Epoch 244
Loss = 3.0169e-01, PNorm = 40.5262, GNorm = 3.4740, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.424516
Epoch 245
Validation binary_cross_entropy = 0.365862
Epoch 246
Validation binary_cross_entropy = 0.343542
Epoch 247
Validation binary_cross_entropy = 0.349258
Epoch 248
Validation binary_cross_entropy = 0.375957
Epoch 249
Loss = 2.0818e-01, PNorm = 40.5588, GNorm = 2.3102, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.396442
Epoch 250
Validation binary_cross_entropy = 0.397835
Epoch 251
Validation binary_cross_entropy = 0.397174
Epoch 252
Validation binary_cross_entropy = 0.393039
Epoch 253
Validation binary_cross_entropy = 0.386736
Epoch 254
Loss = 2.5090e-01, PNorm = 40.5923, GNorm = 1.5842, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.393697
Epoch 255
Validation binary_cross_entropy = 0.402382
Epoch 256
Validation binary_cross_entropy = 0.406095
Epoch 257
Validation binary_cross_entropy = 0.364682
Epoch 258
Validation binary_cross_entropy = 0.345300
Epoch 259
Loss = 4.5691e-01, PNorm = 40.6287, GNorm = 2.5707, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.342430
Epoch 260
Validation binary_cross_entropy = 0.344756
Epoch 261
Validation binary_cross_entropy = 0.364738
Epoch 262
Validation binary_cross_entropy = 0.420251
Epoch 263
Validation binary_cross_entropy = 0.491886
Epoch 264
Loss = 2.6417e-01, PNorm = 40.6636, GNorm = 1.9236, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.538668
Epoch 265
Validation binary_cross_entropy = 0.527372
Epoch 266
Validation binary_cross_entropy = 0.447922
Epoch 267
Validation binary_cross_entropy = 0.377985
Epoch 268
Validation binary_cross_entropy = 0.352600
Epoch 269
Loss = 3.1025e-01, PNorm = 40.6969, GNorm = 3.0870, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.358284
Epoch 270
Validation binary_cross_entropy = 0.368471
Epoch 271
Validation binary_cross_entropy = 0.387948
Epoch 272
Validation binary_cross_entropy = 0.419973
Epoch 273
Validation binary_cross_entropy = 0.414026
Epoch 274
Loss = 2.4073e-01, PNorm = 40.7293, GNorm = 1.9184, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.394744
Epoch 275
Validation binary_cross_entropy = 0.382923
Epoch 276
Validation binary_cross_entropy = 0.382418
Epoch 277
Validation binary_cross_entropy = 0.382791
Epoch 278
Validation binary_cross_entropy = 0.383109
Epoch 279
Loss = 2.2473e-01, PNorm = 40.7618, GNorm = 1.7131, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.392475
Epoch 280
Validation binary_cross_entropy = 0.414401
Epoch 281
Validation binary_cross_entropy = 0.412885
Epoch 282
Validation binary_cross_entropy = 0.416136
Epoch 283
Validation binary_cross_entropy = 0.407111
Epoch 284
Loss = 3.7344e-01, PNorm = 40.7914, GNorm = 3.2793, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.392513
Epoch 285
Validation binary_cross_entropy = 0.371327
Epoch 286
Validation binary_cross_entropy = 0.362424
Epoch 287
Validation binary_cross_entropy = 0.375582
Epoch 288
Validation binary_cross_entropy = 0.398747
Epoch 289
Loss = 2.9199e-01, PNorm = 40.8240, GNorm = 1.9136, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.430704
Epoch 290
Validation binary_cross_entropy = 0.460590
Epoch 291
Validation binary_cross_entropy = 0.490876
Epoch 292
Validation binary_cross_entropy = 0.515540
Epoch 293
Validation binary_cross_entropy = 0.493574
Epoch 294
Loss = 2.4341e-01, PNorm = 40.8580, GNorm = 1.7756, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.456576
Epoch 295
Validation binary_cross_entropy = 0.428959
Epoch 296
Validation binary_cross_entropy = 0.416725
Epoch 297
Validation binary_cross_entropy = 0.393826
Epoch 298
Validation binary_cross_entropy = 0.373707
Epoch 299
Loss = 2.7922e-01, PNorm = 40.8880, GNorm = 2.5429, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.362157
Model 0 best validation binary_cross_entropy = 0.308237 on epoch 21
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.230162
Ensemble test binary_cross_entropy = 0.230162
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 1,303,001
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.315048
Epoch 1
Validation binary_cross_entropy = 0.506204
Epoch 2
Validation binary_cross_entropy = 0.593457
Epoch 3
Validation binary_cross_entropy = 0.424594
Epoch 4
Loss = 9.4875e-01, PNorm = 38.3093, GNorm = 10.0642, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.389084
Epoch 5
Validation binary_cross_entropy = 0.500644
Epoch 6
Validation binary_cross_entropy = 0.337444
Epoch 7
Validation binary_cross_entropy = 0.321359
Epoch 8
Validation binary_cross_entropy = 0.409119
Epoch 9
Loss = 1.0523e+00, PNorm = 38.3938, GNorm = 9.9121, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.316865
Epoch 10
Validation binary_cross_entropy = 0.367552
Epoch 11
Validation binary_cross_entropy = 0.342511
Epoch 12
Validation binary_cross_entropy = 0.289779
Epoch 13
Validation binary_cross_entropy = 0.659547
Epoch 14
Loss = 9.5343e-01, PNorm = 38.5153, GNorm = 6.4899, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.542875
Epoch 15
Validation binary_cross_entropy = 0.420471
Epoch 16
Validation binary_cross_entropy = 0.418309
Epoch 17
Validation binary_cross_entropy = 0.307755
Epoch 18
Validation binary_cross_entropy = 0.345064
Epoch 19
Loss = 6.7951e-01, PNorm = 38.6271, GNorm = 7.2363, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.528367
Epoch 20
Validation binary_cross_entropy = 0.383800
Epoch 21
Validation binary_cross_entropy = 0.356124
Epoch 22
Validation binary_cross_entropy = 0.453651
Epoch 23
Validation binary_cross_entropy = 0.368686
Epoch 24
Loss = 6.2465e-01, PNorm = 38.7170, GNorm = 7.2716, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.329918
Epoch 25
Validation binary_cross_entropy = 0.459172
Epoch 26
Validation binary_cross_entropy = 0.428500
Epoch 27
Validation binary_cross_entropy = 0.337167
Epoch 28
Validation binary_cross_entropy = 0.335777
Epoch 29
Loss = 6.2041e-01, PNorm = 38.7973, GNorm = 5.5453, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.449555
Epoch 30
Validation binary_cross_entropy = 0.458932
Epoch 31
Validation binary_cross_entropy = 0.384314
Epoch 32
Validation binary_cross_entropy = 0.459420
Epoch 33
Validation binary_cross_entropy = 0.566840
Epoch 34
Loss = 7.0152e-01, PNorm = 38.8673, GNorm = 4.7969, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.747459
Epoch 35
Validation binary_cross_entropy = 1.030987
Epoch 36
Validation binary_cross_entropy = 0.689688
Epoch 37
Validation binary_cross_entropy = 0.375994
Epoch 38
Validation binary_cross_entropy = 0.329935
Epoch 39
Loss = 6.9162e-01, PNorm = 38.9257, GNorm = 6.4087, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.532379
Epoch 40
Validation binary_cross_entropy = 0.873073
Epoch 41
Validation binary_cross_entropy = 0.514875
Epoch 42
Validation binary_cross_entropy = 0.317923
Epoch 43
Validation binary_cross_entropy = 0.345116
Epoch 44
Loss = 6.5334e-01, PNorm = 38.9796, GNorm = 3.6000, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.568679
Epoch 45
Validation binary_cross_entropy = 0.957790
Epoch 46
Validation binary_cross_entropy = 0.751645
Epoch 47
Validation binary_cross_entropy = 0.508527
Epoch 48
Validation binary_cross_entropy = 0.468719
Epoch 49
Loss = 7.4107e-01, PNorm = 39.0300, GNorm = 4.9516, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.506879
Epoch 50
Validation binary_cross_entropy = 0.703222
Epoch 51
Validation binary_cross_entropy = 0.613058
Epoch 52
Validation binary_cross_entropy = 0.462734
Epoch 53
Validation binary_cross_entropy = 0.418449
Epoch 54
Loss = 5.6801e-01, PNorm = 39.0787, GNorm = 4.5668, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.423239
Epoch 55
Validation binary_cross_entropy = 0.470833
Epoch 56
Validation binary_cross_entropy = 0.535844
Epoch 57
Validation binary_cross_entropy = 0.539963
Epoch 58
Validation binary_cross_entropy = 0.481011
Epoch 59
Loss = 6.3032e-01, PNorm = 39.1233, GNorm = 4.4554, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.481804
Epoch 60
Validation binary_cross_entropy = 0.442575
Epoch 61
Validation binary_cross_entropy = 0.471120
Epoch 62
Validation binary_cross_entropy = 0.548802
Epoch 63
Validation binary_cross_entropy = 0.563769
Epoch 64
Loss = 4.8091e-01, PNorm = 39.1656, GNorm = 2.9482, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.591018
Epoch 65
Validation binary_cross_entropy = 0.578776
Epoch 66
Validation binary_cross_entropy = 0.474896
Epoch 67
Validation binary_cross_entropy = 0.374801
Epoch 68
Validation binary_cross_entropy = 0.365701
Epoch 69
Loss = 4.9140e-01, PNorm = 39.2053, GNorm = 3.0932, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.467105
Epoch 70
Validation binary_cross_entropy = 0.574888
Epoch 71
Validation binary_cross_entropy = 0.552713
Epoch 72
Validation binary_cross_entropy = 0.455164
Epoch 73
Validation binary_cross_entropy = 0.428737
Epoch 74
Loss = 5.7859e-01, PNorm = 39.2454, GNorm = 3.8985, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.432138
Epoch 75
Validation binary_cross_entropy = 0.476448
Epoch 76
Validation binary_cross_entropy = 0.527616
Epoch 77
Validation binary_cross_entropy = 0.548546
Epoch 78
Validation binary_cross_entropy = 0.481491
Epoch 79
Loss = 4.4756e-01, PNorm = 39.2839, GNorm = 3.6630, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.392324
Epoch 80
Validation binary_cross_entropy = 0.391511
Epoch 81
Validation binary_cross_entropy = 0.431315
Epoch 82
Validation binary_cross_entropy = 0.502683
Epoch 83
Validation binary_cross_entropy = 0.568163
Epoch 84
Loss = 5.1976e-01, PNorm = 39.3244, GNorm = 3.6748, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.507699
Epoch 85
Validation binary_cross_entropy = 0.415636
Epoch 86
Validation binary_cross_entropy = 0.410461
Epoch 87
Validation binary_cross_entropy = 0.425751
Epoch 88
Validation binary_cross_entropy = 0.459042
Epoch 89
Loss = 4.4899e-01, PNorm = 39.3659, GNorm = 2.4593, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.472186
Epoch 90
Validation binary_cross_entropy = 0.465111
Epoch 91
Validation binary_cross_entropy = 0.450756
Epoch 92
Validation binary_cross_entropy = 0.428939
Epoch 93
Validation binary_cross_entropy = 0.411899
Epoch 94
Loss = 4.9738e-01, PNorm = 39.4045, GNorm = 4.0511, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.450658
Epoch 95
Validation binary_cross_entropy = 0.536997
Epoch 96
Validation binary_cross_entropy = 0.586445
Epoch 97
Validation binary_cross_entropy = 0.539655
Epoch 98
Validation binary_cross_entropy = 0.460705
Epoch 99
Loss = 4.3529e-01, PNorm = 39.4414, GNorm = 2.5099, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.363921
Epoch 100
Validation binary_cross_entropy = 0.375258
Epoch 101
Validation binary_cross_entropy = 0.420263
Epoch 102
Validation binary_cross_entropy = 0.465436
Epoch 103
Validation binary_cross_entropy = 0.537528
Epoch 104
Loss = 3.8840e-01, PNorm = 39.4804, GNorm = 3.2672, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.583618
Epoch 105
Validation binary_cross_entropy = 0.572243
Epoch 106
Validation binary_cross_entropy = 0.505818
Epoch 107
Validation binary_cross_entropy = 0.403437
Epoch 108
Validation binary_cross_entropy = 0.355585
Epoch 109
Loss = 5.4875e-01, PNorm = 39.5191, GNorm = 3.0440, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.385966
Epoch 110
Validation binary_cross_entropy = 0.491315
Epoch 111
Validation binary_cross_entropy = 0.588736
Epoch 112
Validation binary_cross_entropy = 0.535964
Epoch 113
Validation binary_cross_entropy = 0.464031
Epoch 114
Loss = 4.3014e-01, PNorm = 39.5571, GNorm = 2.1232, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.412159
Epoch 115
Validation binary_cross_entropy = 0.411787
Epoch 116
Validation binary_cross_entropy = 0.420611
Epoch 117
Validation binary_cross_entropy = 0.440241
Epoch 118
Validation binary_cross_entropy = 0.476342
Epoch 119
Loss = 4.4432e-01, PNorm = 39.5969, GNorm = 2.7554, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.476053
Epoch 120
Validation binary_cross_entropy = 0.467909
Epoch 121
Validation binary_cross_entropy = 0.443908
Epoch 122
Validation binary_cross_entropy = 0.411493
Epoch 123
Validation binary_cross_entropy = 0.396048
Epoch 124
Loss = 3.5467e-01, PNorm = 39.6359, GNorm = 2.8668, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.409942
Epoch 125
Validation binary_cross_entropy = 0.482406
Epoch 126
Validation binary_cross_entropy = 0.456106
Epoch 127
Validation binary_cross_entropy = 0.465979
Epoch 128
Validation binary_cross_entropy = 0.426038
Epoch 129
Loss = 3.2855e-01, PNorm = 39.6775, GNorm = 2.2350, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.407388
Epoch 130
Validation binary_cross_entropy = 0.435733
Epoch 131
Validation binary_cross_entropy = 0.476961
Epoch 132
Validation binary_cross_entropy = 0.544764
Epoch 133
Validation binary_cross_entropy = 0.505606
Epoch 134
Loss = 4.4071e-01, PNorm = 39.7171, GNorm = 2.5614, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.448027
Epoch 135
Validation binary_cross_entropy = 0.432195
Epoch 136
Validation binary_cross_entropy = 0.461471
Epoch 137
Validation binary_cross_entropy = 0.482744
Epoch 138
Validation binary_cross_entropy = 0.475063
Epoch 139
Loss = 3.9895e-01, PNorm = 39.7586, GNorm = 3.3839, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.465473
Epoch 140
Validation binary_cross_entropy = 0.442596
Epoch 141
Validation binary_cross_entropy = 0.438694
Epoch 142
Validation binary_cross_entropy = 0.455627
Epoch 143
Validation binary_cross_entropy = 0.470117
Epoch 144
Loss = 4.4727e-01, PNorm = 39.7996, GNorm = 2.7797, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.488645
Epoch 145
Validation binary_cross_entropy = 0.478022
Epoch 146
Validation binary_cross_entropy = 0.451314
Epoch 147
Validation binary_cross_entropy = 0.432515
Epoch 148
Validation binary_cross_entropy = 0.380918
Epoch 149
Loss = 4.7384e-01, PNorm = 39.8400, GNorm = 2.8440, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.391140
Epoch 150
Validation binary_cross_entropy = 0.459533
Epoch 151
Validation binary_cross_entropy = 0.525562
Epoch 152
Validation binary_cross_entropy = 0.552283
Epoch 153
Validation binary_cross_entropy = 0.512271
Epoch 154
Loss = 4.7064e-01, PNorm = 39.8824, GNorm = 2.8355, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.414592
Epoch 155
Validation binary_cross_entropy = 0.407007
Epoch 156
Validation binary_cross_entropy = 0.485505
Epoch 157
Validation binary_cross_entropy = 0.598692
Epoch 158
Validation binary_cross_entropy = 0.657833
Epoch 159
Loss = 3.8031e-01, PNorm = 39.9247, GNorm = 2.8315, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.540696
Epoch 160
Validation binary_cross_entropy = 0.409281
Epoch 161
Validation binary_cross_entropy = 0.367063
Epoch 162
Validation binary_cross_entropy = 0.391193
Epoch 163
Validation binary_cross_entropy = 0.469211
Epoch 164
Loss = 4.2082e-01, PNorm = 39.9647, GNorm = 2.7315, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.532149
Epoch 165
Validation binary_cross_entropy = 0.601301
Epoch 166
Validation binary_cross_entropy = 0.659271
Epoch 167
Validation binary_cross_entropy = 0.730574
Epoch 168
Validation binary_cross_entropy = 0.741938
Epoch 169
Loss = 4.2491e-01, PNorm = 40.0019, GNorm = 2.7677, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.662397
Epoch 170
Validation binary_cross_entropy = 0.572750
Epoch 171
Validation binary_cross_entropy = 0.481866
Epoch 172
Validation binary_cross_entropy = 0.453132
Epoch 173
Validation binary_cross_entropy = 0.509968
Epoch 174
Loss = 4.1929e-01, PNorm = 40.0383, GNorm = 2.4281, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.484327
Epoch 175
Validation binary_cross_entropy = 0.523253
Epoch 176
Validation binary_cross_entropy = 0.544536
Epoch 177
Validation binary_cross_entropy = 0.490273
Epoch 178
Validation binary_cross_entropy = 0.455968
Epoch 179
Loss = 4.4399e-01, PNorm = 40.0769, GNorm = 2.7798, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.462547
Epoch 180
Validation binary_cross_entropy = 0.478373
Epoch 181
Validation binary_cross_entropy = 0.505687
Epoch 182
Validation binary_cross_entropy = 0.512387
Epoch 183
Validation binary_cross_entropy = 0.479208
Epoch 184
Loss = 3.7205e-01, PNorm = 40.1147, GNorm = 2.2604, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.452302
Epoch 185
Validation binary_cross_entropy = 0.427399
Epoch 186
Validation binary_cross_entropy = 0.408110
Epoch 187
Validation binary_cross_entropy = 0.446734
Epoch 188
Validation binary_cross_entropy = 0.493794
Epoch 189
Loss = 3.5495e-01, PNorm = 40.1510, GNorm = 3.0994, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.513930
Epoch 190
Validation binary_cross_entropy = 0.537561
Epoch 191
Validation binary_cross_entropy = 0.501691
Epoch 192
Validation binary_cross_entropy = 0.480794
Epoch 193
Validation binary_cross_entropy = 0.417475
Epoch 194
Loss = 3.9919e-01, PNorm = 40.1900, GNorm = 2.3985, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.374123
Epoch 195
Validation binary_cross_entropy = 0.379574
Epoch 196
Validation binary_cross_entropy = 0.424769
Epoch 197
Validation binary_cross_entropy = 0.463096
Epoch 198
Validation binary_cross_entropy = 0.479841
Epoch 199
Loss = 3.5107e-01, PNorm = 40.2264, GNorm = 2.1670, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.478020
Epoch 200
Validation binary_cross_entropy = 0.437511
Epoch 201
Validation binary_cross_entropy = 0.411289
Epoch 202
Validation binary_cross_entropy = 0.421188
Epoch 203
Validation binary_cross_entropy = 0.449360
Epoch 204
Loss = 2.9350e-01, PNorm = 40.2619, GNorm = 2.3717, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.493033
Epoch 205
Validation binary_cross_entropy = 0.510012
Epoch 206
Validation binary_cross_entropy = 0.476211
Epoch 207
Validation binary_cross_entropy = 0.390045
Epoch 208
Validation binary_cross_entropy = 0.363669
Epoch 209
Loss = 4.2382e-01, PNorm = 40.2980, GNorm = 3.1982, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.371540
Epoch 210
Validation binary_cross_entropy = 0.401371
Epoch 211
Validation binary_cross_entropy = 0.415053
Epoch 212
Validation binary_cross_entropy = 0.416744
Epoch 213
Validation binary_cross_entropy = 0.432903
Epoch 214
Loss = 4.2018e-01, PNorm = 40.3322, GNorm = 3.3339, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.459426
Epoch 215
Validation binary_cross_entropy = 0.449451
Epoch 216
Validation binary_cross_entropy = 0.411762
Epoch 217
Validation binary_cross_entropy = 0.392801
Epoch 218
Validation binary_cross_entropy = 0.394454
Epoch 219
Loss = 3.6615e-01, PNorm = 40.3737, GNorm = 2.1900, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.387790
Epoch 220
Validation binary_cross_entropy = 0.402517
Epoch 221
Validation binary_cross_entropy = 0.426265
Epoch 222
Validation binary_cross_entropy = 0.452719
Epoch 223
Validation binary_cross_entropy = 0.439261
Epoch 224
Loss = 3.4898e-01, PNorm = 40.4122, GNorm = 2.6412, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.442288
Epoch 225
Validation binary_cross_entropy = 0.431940
Epoch 226
Validation binary_cross_entropy = 0.466769
Epoch 227
Validation binary_cross_entropy = 0.504871
Epoch 228
Validation binary_cross_entropy = 0.532800
Epoch 229
Loss = 4.0242e-01, PNorm = 40.4464, GNorm = 2.8100, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.470359
Epoch 230
Validation binary_cross_entropy = 0.425802
Epoch 231
Validation binary_cross_entropy = 0.425186
Epoch 232
Validation binary_cross_entropy = 0.474617
Epoch 233
Validation binary_cross_entropy = 0.507957
Epoch 234
Loss = 3.4240e-01, PNorm = 40.4831, GNorm = 2.0763, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.473689
Epoch 235
Validation binary_cross_entropy = 0.402813
Epoch 236
Validation binary_cross_entropy = 0.383887
Epoch 237
Validation binary_cross_entropy = 0.392439
Epoch 238
Validation binary_cross_entropy = 0.406030
Epoch 239
Loss = 3.5016e-01, PNorm = 40.5140, GNorm = 2.2188, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.420403
Epoch 240
Validation binary_cross_entropy = 0.419277
Epoch 241
Validation binary_cross_entropy = 0.439987
Epoch 242
Validation binary_cross_entropy = 0.463577
Epoch 243
Validation binary_cross_entropy = 0.437082
Epoch 244
Loss = 3.4537e-01, PNorm = 40.5474, GNorm = 3.3769, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.397129
Epoch 245
Validation binary_cross_entropy = 0.400506
Epoch 246
Validation binary_cross_entropy = 0.401958
Epoch 247
Validation binary_cross_entropy = 0.426668
Epoch 248
Validation binary_cross_entropy = 0.467687
Epoch 249
Loss = 3.3609e-01, PNorm = 40.5839, GNorm = 3.2727, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.487502
Epoch 250
Validation binary_cross_entropy = 0.472835
Epoch 251
Validation binary_cross_entropy = 0.435291
Epoch 252
Validation binary_cross_entropy = 0.412244
Epoch 253
Validation binary_cross_entropy = 0.386005
Epoch 254
Loss = 2.8923e-01, PNorm = 40.6173, GNorm = 2.4319, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.373770
Epoch 255
Validation binary_cross_entropy = 0.377496
Epoch 256
Validation binary_cross_entropy = 0.369031
Epoch 257
Validation binary_cross_entropy = 0.367613
Epoch 258
Validation binary_cross_entropy = 0.395806
Epoch 259
Loss = 2.9574e-01, PNorm = 40.6485, GNorm = 1.6719, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.424448
Epoch 260
Validation binary_cross_entropy = 0.412531
Epoch 261
Validation binary_cross_entropy = 0.415225
Epoch 262
Validation binary_cross_entropy = 0.453955
Epoch 263
Validation binary_cross_entropy = 0.512092
Epoch 264
Loss = 2.9689e-01, PNorm = 40.6792, GNorm = 2.5410, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.576609
Epoch 265
Validation binary_cross_entropy = 0.589023
Epoch 266
Validation binary_cross_entropy = 0.629399
Epoch 267
Validation binary_cross_entropy = 0.565014
Epoch 268
Validation binary_cross_entropy = 0.505750
Epoch 269
Loss = 3.5407e-01, PNorm = 40.7118, GNorm = 3.7554, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.460120
Epoch 270
Validation binary_cross_entropy = 0.425601
Epoch 271
Validation binary_cross_entropy = 0.385075
Epoch 272
Validation binary_cross_entropy = 0.366633
Epoch 273
Validation binary_cross_entropy = 0.367306
Epoch 274
Loss = 3.9112e-01, PNorm = 40.7419, GNorm = 3.1568, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.380167
Epoch 275
Validation binary_cross_entropy = 0.399774
Epoch 276
Validation binary_cross_entropy = 0.391124
Epoch 277
Validation binary_cross_entropy = 0.374069
Epoch 278
Validation binary_cross_entropy = 0.353894
Epoch 279
Loss = 2.6243e-01, PNorm = 40.7688, GNorm = 2.9779, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.345673
Epoch 280
Validation binary_cross_entropy = 0.355693
Epoch 281
Validation binary_cross_entropy = 0.368618
Epoch 282
Validation binary_cross_entropy = 0.408254
Epoch 283
Validation binary_cross_entropy = 0.441564
Epoch 284
Loss = 3.2126e-01, PNorm = 40.7989, GNorm = 2.7981, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.433604
Epoch 285
Validation binary_cross_entropy = 0.431178
Epoch 286
Validation binary_cross_entropy = 0.403425
Epoch 287
Validation binary_cross_entropy = 0.402597
Epoch 288
Validation binary_cross_entropy = 0.406787
Epoch 289
Loss = 3.0022e-01, PNorm = 40.8281, GNorm = 2.3609, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.425036
Epoch 290
Validation binary_cross_entropy = 0.475429
Epoch 291
Validation binary_cross_entropy = 0.469894
Epoch 292
Validation binary_cross_entropy = 0.438436
Epoch 293
Validation binary_cross_entropy = 0.386926
Epoch 294
Loss = 1.9835e-01, PNorm = 40.8598, GNorm = 1.8661, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.360961
Epoch 295
Validation binary_cross_entropy = 0.350517
Epoch 296
Validation binary_cross_entropy = 0.339993
Epoch 297
Validation binary_cross_entropy = 0.340551
Epoch 298
Validation binary_cross_entropy = 0.343058
Epoch 299
Loss = 1.9868e-01, PNorm = 40.8938, GNorm = 2.0671, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.350459
Model 0 best validation binary_cross_entropy = 0.289779 on epoch 12
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.209275
Ensemble test binary_cross_entropy = 0.209275
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 1,303,001
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.314938
Epoch 1
Validation binary_cross_entropy = 0.729936
Epoch 2
Validation binary_cross_entropy = 0.812999
Epoch 3
Validation binary_cross_entropy = 0.327337
Epoch 4
Loss = 8.8753e-01, PNorm = 38.3094, GNorm = 7.6261, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.369079
Epoch 5
Validation binary_cross_entropy = 0.768870
Epoch 6
Validation binary_cross_entropy = 0.445986
Epoch 7
Validation binary_cross_entropy = 0.294845
Epoch 8
Validation binary_cross_entropy = 0.278881
Epoch 9
Loss = 8.8477e-01, PNorm = 38.3971, GNorm = 8.2585, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.469207
Epoch 10
Validation binary_cross_entropy = 0.406024
Epoch 11
Validation binary_cross_entropy = 0.281230
Epoch 12
Validation binary_cross_entropy = 0.457294
Epoch 13
Validation binary_cross_entropy = 0.655252
Epoch 14
Loss = 8.9480e-01, PNorm = 38.5118, GNorm = 7.7579, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.383279
Epoch 15
Validation binary_cross_entropy = 0.547476
Epoch 16
Validation binary_cross_entropy = 1.291863
Epoch 17
Validation binary_cross_entropy = 0.446405
Epoch 18
Validation binary_cross_entropy = 0.277299
Epoch 19
Loss = 7.5463e-01, PNorm = 38.6148, GNorm = 7.9164, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.461932
Epoch 20
Validation binary_cross_entropy = 0.898629
Epoch 21
Validation binary_cross_entropy = 0.403266
Epoch 22
Validation binary_cross_entropy = 0.310557
Epoch 23
Validation binary_cross_entropy = 0.393787
Epoch 24
Loss = 7.3237e-01, PNorm = 38.7053, GNorm = 6.4328, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.479564
Epoch 25
Validation binary_cross_entropy = 0.517977
Epoch 26
Validation binary_cross_entropy = 0.331191
Epoch 27
Validation binary_cross_entropy = 0.330637
Epoch 28
Validation binary_cross_entropy = 0.453716
Epoch 29
Loss = 6.9161e-01, PNorm = 38.7802, GNorm = 7.2624, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.447744
Epoch 30
Validation binary_cross_entropy = 0.332628
Epoch 31
Validation binary_cross_entropy = 0.334230
Epoch 32
Validation binary_cross_entropy = 0.659463
Epoch 33
Validation binary_cross_entropy = 0.931582
Epoch 34
Loss = 5.0822e-01, PNorm = 38.8486, GNorm = 3.9052, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.575595
Epoch 35
Validation binary_cross_entropy = 0.356661
Epoch 36
Validation binary_cross_entropy = 0.348420
Epoch 37
Validation binary_cross_entropy = 0.432311
Epoch 38
Validation binary_cross_entropy = 0.676265
Epoch 39
Loss = 7.1360e-01, PNorm = 38.9052, GNorm = 6.4403, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.574254
Epoch 40
Validation binary_cross_entropy = 0.447172
Epoch 41
Validation binary_cross_entropy = 0.613002
Epoch 42
Validation binary_cross_entropy = 1.080526
Epoch 43
Validation binary_cross_entropy = 1.213270
Epoch 44
Loss = 5.3535e-01, PNorm = 38.9592, GNorm = 4.3325, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.884239
Epoch 45
Validation binary_cross_entropy = 0.547740
Epoch 46
Validation binary_cross_entropy = 0.415464
Epoch 47
Validation binary_cross_entropy = 0.454914
Epoch 48
Validation binary_cross_entropy = 0.610427
Epoch 49
Loss = 5.3975e-01, PNorm = 39.0100, GNorm = 4.8689, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.526788
Epoch 50
Validation binary_cross_entropy = 0.420348
Epoch 51
Validation binary_cross_entropy = 0.368885
Epoch 52
Validation binary_cross_entropy = 0.421687
Epoch 53
Validation binary_cross_entropy = 0.607253
Epoch 54
Loss = 4.9741e-01, PNorm = 39.0588, GNorm = 3.5063, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.704015
Epoch 55
Validation binary_cross_entropy = 0.596096
Epoch 56
Validation binary_cross_entropy = 0.703390
Epoch 57
Validation binary_cross_entropy = 0.755594
Epoch 58
Validation binary_cross_entropy = 0.833840
Epoch 59
Loss = 4.5001e-01, PNorm = 39.1092, GNorm = 2.7288, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.819914
Epoch 60
Validation binary_cross_entropy = 0.471316
Epoch 61
Validation binary_cross_entropy = 0.382828
Epoch 62
Validation binary_cross_entropy = 0.469822
Epoch 63
Validation binary_cross_entropy = 0.606814
Epoch 64
Loss = 6.4767e-01, PNorm = 39.1570, GNorm = 4.4058, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.666928
Epoch 65
Validation binary_cross_entropy = 0.549694
Epoch 66
Validation binary_cross_entropy = 0.439860
Epoch 67
Validation binary_cross_entropy = 0.391634
Epoch 68
Validation binary_cross_entropy = 0.459655
Epoch 69
Loss = 4.0304e-01, PNorm = 39.2007, GNorm = 2.7648, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.625163
Epoch 70
Validation binary_cross_entropy = 0.697681
Epoch 71
Validation binary_cross_entropy = 0.756811
Epoch 72
Validation binary_cross_entropy = 0.660956
Epoch 73
Validation binary_cross_entropy = 0.505933
Epoch 74
Loss = 5.8415e-01, PNorm = 39.2444, GNorm = 3.6074, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.474168
Epoch 75
Validation binary_cross_entropy = 0.475602
Epoch 76
Validation binary_cross_entropy = 0.498723
Epoch 77
Validation binary_cross_entropy = 0.449072
Epoch 78
Validation binary_cross_entropy = 0.412465
Epoch 79
Loss = 4.5304e-01, PNorm = 39.2855, GNorm = 3.0037, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.507399
Epoch 80
Validation binary_cross_entropy = 0.617184
Epoch 81
Validation binary_cross_entropy = 0.542202
Epoch 82
Validation binary_cross_entropy = 0.418532
Epoch 83
Validation binary_cross_entropy = 0.343580
Epoch 84
Loss = 5.3875e-01, PNorm = 39.3289, GNorm = 4.4325, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.379220
Epoch 85
Validation binary_cross_entropy = 0.473683
Epoch 86
Validation binary_cross_entropy = 0.497310
Epoch 87
Validation binary_cross_entropy = 0.499647
Epoch 88
Validation binary_cross_entropy = 0.471802
Epoch 89
Loss = 4.8657e-01, PNorm = 39.3735, GNorm = 3.5666, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.459220
Epoch 90
Validation binary_cross_entropy = 0.472736
Epoch 91
Validation binary_cross_entropy = 0.478937
Epoch 92
Validation binary_cross_entropy = 0.452490
Epoch 93
Validation binary_cross_entropy = 0.431779
Epoch 94
Loss = 4.0308e-01, PNorm = 39.4182, GNorm = 2.6569, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.421652
Epoch 95
Validation binary_cross_entropy = 0.386298
Epoch 96
Validation binary_cross_entropy = 0.370867
Epoch 97
Validation binary_cross_entropy = 0.378849
Epoch 98
Validation binary_cross_entropy = 0.462500
Epoch 99
Loss = 3.8857e-01, PNorm = 39.4624, GNorm = 3.0158, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.518610
Epoch 100
Validation binary_cross_entropy = 0.433000
Epoch 101
Validation binary_cross_entropy = 0.365188
Epoch 102
Validation binary_cross_entropy = 0.360651
Epoch 103
Validation binary_cross_entropy = 0.421988
Epoch 104
Loss = 4.4478e-01, PNorm = 39.5136, GNorm = 2.9329, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.497539
Epoch 105
Validation binary_cross_entropy = 0.523211
Epoch 106
Validation binary_cross_entropy = 0.417747
Epoch 107
Validation binary_cross_entropy = 0.325407
Epoch 108
Validation binary_cross_entropy = 0.331161
Epoch 109
Loss = 4.3635e-01, PNorm = 39.5620, GNorm = 4.2732, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.457686
Epoch 110
Validation binary_cross_entropy = 0.690855
Epoch 111
Validation binary_cross_entropy = 0.877076
Epoch 112
Validation binary_cross_entropy = 0.610440
Epoch 113
Validation binary_cross_entropy = 0.389174
Epoch 114
Loss = 3.2646e-01, PNorm = 39.6112, GNorm = 3.2869, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.329705
Epoch 115
Validation binary_cross_entropy = 0.356067
Epoch 116
Validation binary_cross_entropy = 0.452378
Epoch 117
Validation binary_cross_entropy = 0.623074
Epoch 118
Validation binary_cross_entropy = 0.737545
Epoch 119
Loss = 3.9443e-01, PNorm = 39.6586, GNorm = 2.6390, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.689324
Epoch 120
Validation binary_cross_entropy = 0.578393
Epoch 121
Validation binary_cross_entropy = 0.448536
Epoch 122
Validation binary_cross_entropy = 0.355818
Epoch 123
Validation binary_cross_entropy = 0.366444
Epoch 124
Loss = 4.3226e-01, PNorm = 39.7047, GNorm = 3.0788, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.472798
Epoch 125
Validation binary_cross_entropy = 0.598023
Epoch 126
Validation binary_cross_entropy = 0.621910
Epoch 127
Validation binary_cross_entropy = 0.556698
Epoch 128
Validation binary_cross_entropy = 0.462509
Epoch 129
Loss = 3.6456e-01, PNorm = 39.7505, GNorm = 2.8023, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.420891
Epoch 130
Validation binary_cross_entropy = 0.416066
Epoch 131
Validation binary_cross_entropy = 0.451185
Epoch 132
Validation binary_cross_entropy = 0.506046
Epoch 133
Validation binary_cross_entropy = 0.637708
Epoch 134
Loss = 4.5088e-01, PNorm = 39.7901, GNorm = 3.1528, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.640672
Epoch 135
Validation binary_cross_entropy = 0.674496
Epoch 136
Validation binary_cross_entropy = 0.700858
Epoch 137
Validation binary_cross_entropy = 0.675585
Epoch 138
Validation binary_cross_entropy = 0.638660
Epoch 139
Loss = 4.1619e-01, PNorm = 39.8322, GNorm = 2.7297, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.650239
Epoch 140
Validation binary_cross_entropy = 0.711396
Epoch 141
Validation binary_cross_entropy = 0.768601
Epoch 142
Validation binary_cross_entropy = 0.665344
Epoch 143
Validation binary_cross_entropy = 0.442614
Epoch 144
Loss = 5.6213e-01, PNorm = 39.8735, GNorm = 2.7989, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.332598
Epoch 145
Validation binary_cross_entropy = 0.339807
Epoch 146
Validation binary_cross_entropy = 0.422667
Epoch 147
Validation binary_cross_entropy = 0.443447
Epoch 148
Validation binary_cross_entropy = 0.431638
Epoch 149
Loss = 4.4256e-01, PNorm = 39.9143, GNorm = 2.8455, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.479839
Epoch 150
Validation binary_cross_entropy = 0.575237
Epoch 151
Validation binary_cross_entropy = 0.621049
Epoch 152
Validation binary_cross_entropy = 0.527276
Epoch 153
Validation binary_cross_entropy = 0.438802
Epoch 154
Loss = 3.4025e-01, PNorm = 39.9572, GNorm = 2.8627, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.438111
Epoch 155
Validation binary_cross_entropy = 0.500078
Epoch 156
Validation binary_cross_entropy = 0.557716
Epoch 157
Validation binary_cross_entropy = 0.508859
Epoch 158
Validation binary_cross_entropy = 0.415874
Epoch 159
Loss = 3.5117e-01, PNorm = 40.0007, GNorm = 2.4268, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.353487
Epoch 160
Validation binary_cross_entropy = 0.356097
Epoch 161
Validation binary_cross_entropy = 0.391240
Epoch 162
Validation binary_cross_entropy = 0.461953
Epoch 163
Validation binary_cross_entropy = 0.475714
Epoch 164
Loss = 4.2912e-01, PNorm = 40.0419, GNorm = 3.4653, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.448166
Epoch 165
Validation binary_cross_entropy = 0.426192
Epoch 166
Validation binary_cross_entropy = 0.391865
Epoch 167
Validation binary_cross_entropy = 0.403221
Epoch 168
Validation binary_cross_entropy = 0.433355
Epoch 169
Loss = 3.3089e-01, PNorm = 40.0844, GNorm = 2.4969, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.447364
Epoch 170
Validation binary_cross_entropy = 0.459148
Epoch 171
Validation binary_cross_entropy = 0.455588
Epoch 172
Validation binary_cross_entropy = 0.442338
Epoch 173
Validation binary_cross_entropy = 0.472795
Epoch 174
Loss = 4.0578e-01, PNorm = 40.1283, GNorm = 2.7786, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.509443
Epoch 175
Validation binary_cross_entropy = 0.555888
Epoch 176
Validation binary_cross_entropy = 0.537378
Epoch 177
Validation binary_cross_entropy = 0.493979
Epoch 178
Validation binary_cross_entropy = 0.444637
Epoch 179
Loss = 3.3092e-01, PNorm = 40.1695, GNorm = 1.6625, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.418068
Epoch 180
Validation binary_cross_entropy = 0.409877
Epoch 181
Validation binary_cross_entropy = 0.416548
Epoch 182
Validation binary_cross_entropy = 0.433559
Epoch 183
Validation binary_cross_entropy = 0.476314
Epoch 184
Loss = 3.6137e-01, PNorm = 40.2044, GNorm = 1.9449, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.506172
Epoch 185
Validation binary_cross_entropy = 0.557636
Epoch 186
Validation binary_cross_entropy = 0.499854
Epoch 187
Validation binary_cross_entropy = 0.491699
Epoch 188
Validation binary_cross_entropy = 0.480915
Epoch 189
Loss = 3.0125e-01, PNorm = 40.2423, GNorm = 2.1589, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.485864
Epoch 190
Validation binary_cross_entropy = 0.487131
Epoch 191
Validation binary_cross_entropy = 0.493836
Epoch 192
Validation binary_cross_entropy = 0.477852
Epoch 193
Validation binary_cross_entropy = 0.466202
Epoch 194
Loss = 3.1750e-01, PNorm = 40.2773, GNorm = 2.0802, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.441362
Epoch 195
Validation binary_cross_entropy = 0.405365
Epoch 196
Validation binary_cross_entropy = 0.391955
Epoch 197
Validation binary_cross_entropy = 0.440657
Epoch 198
Validation binary_cross_entropy = 0.500562
Epoch 199
Loss = 3.8967e-01, PNorm = 40.3093, GNorm = 4.3631, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.468710
Epoch 200
Validation binary_cross_entropy = 0.413896
Epoch 201
Validation binary_cross_entropy = 0.403543
Epoch 202
Validation binary_cross_entropy = 0.392096
Epoch 203
Validation binary_cross_entropy = 0.396426
Epoch 204
Loss = 4.0379e-01, PNorm = 40.3472, GNorm = 3.6714, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.414040
Epoch 205
Validation binary_cross_entropy = 0.432746
Epoch 206
Validation binary_cross_entropy = 0.440448
Epoch 207
Validation binary_cross_entropy = 0.429507
Epoch 208
Validation binary_cross_entropy = 0.413748
Epoch 209
Loss = 2.4551e-01, PNorm = 40.3846, GNorm = 2.5941, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.391998
Epoch 210
Validation binary_cross_entropy = 0.389869
Epoch 211
Validation binary_cross_entropy = 0.376933
Epoch 212
Validation binary_cross_entropy = 0.383773
Epoch 213
Validation binary_cross_entropy = 0.384755
Epoch 214
Loss = 4.4492e-01, PNorm = 40.4176, GNorm = 3.2063, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.398890
Epoch 215
Validation binary_cross_entropy = 0.435906
Epoch 216
Validation binary_cross_entropy = 0.431271
Epoch 217
Validation binary_cross_entropy = 0.413237
Epoch 218
Validation binary_cross_entropy = 0.397357
Epoch 219
Loss = 3.8995e-01, PNorm = 40.4533, GNorm = 3.0022, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.386109
Epoch 220
Validation binary_cross_entropy = 0.394475
Epoch 221
Validation binary_cross_entropy = 0.411082
Epoch 222
Validation binary_cross_entropy = 0.431147
Epoch 223
Validation binary_cross_entropy = 0.417660
Epoch 224
Loss = 3.2997e-01, PNorm = 40.4917, GNorm = 2.5297, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.382251
Epoch 225
Validation binary_cross_entropy = 0.364657
Epoch 226
Validation binary_cross_entropy = 0.373229
Epoch 227
Validation binary_cross_entropy = 0.409847
Epoch 228
Validation binary_cross_entropy = 0.430407
Epoch 229
Loss = 2.8273e-01, PNorm = 40.5313, GNorm = 2.1342, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.408306
Epoch 230
Validation binary_cross_entropy = 0.376025
Epoch 231
Validation binary_cross_entropy = 0.354832
Epoch 232
Validation binary_cross_entropy = 0.354565
Epoch 233
Validation binary_cross_entropy = 0.370433
Epoch 234
Loss = 4.1536e-01, PNorm = 40.5682, GNorm = 2.4404, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.410061
Epoch 235
Validation binary_cross_entropy = 0.487620
Epoch 236
Validation binary_cross_entropy = 0.469158
Epoch 237
Validation binary_cross_entropy = 0.440359
Epoch 238
Validation binary_cross_entropy = 0.403158
Epoch 239
Loss = 2.8653e-01, PNorm = 40.6041, GNorm = 2.5192, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.376145
Epoch 240
Validation binary_cross_entropy = 0.346856
Epoch 241
Validation binary_cross_entropy = 0.355537
Epoch 242
Validation binary_cross_entropy = 0.382455
Epoch 243
Validation binary_cross_entropy = 0.420415
Epoch 244
Loss = 3.8335e-01, PNorm = 40.6444, GNorm = 3.2993, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.421603
Epoch 245
Validation binary_cross_entropy = 0.381171
Epoch 246
Validation binary_cross_entropy = 0.364245
Epoch 247
Validation binary_cross_entropy = 0.356475
Epoch 248
Validation binary_cross_entropy = 0.368672
Epoch 249
Loss = 2.4536e-01, PNorm = 40.6819, GNorm = 2.4925, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.405766
Epoch 250
Validation binary_cross_entropy = 0.456934
Epoch 251
Validation binary_cross_entropy = 0.493061
Epoch 252
Validation binary_cross_entropy = 0.466327
Epoch 253
Validation binary_cross_entropy = 0.405108
Epoch 254
Loss = 2.7878e-01, PNorm = 40.7210, GNorm = 2.1095, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.370483
Epoch 255
Validation binary_cross_entropy = 0.373982
Epoch 256
Validation binary_cross_entropy = 0.394756
Epoch 257
Validation binary_cross_entropy = 0.452306
Epoch 258
Validation binary_cross_entropy = 0.484054
Epoch 259
Loss = 3.1359e-01, PNorm = 40.7570, GNorm = 2.3112, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.507392
Epoch 260
Validation binary_cross_entropy = 0.496775
Epoch 261
Validation binary_cross_entropy = 0.454790
Epoch 262
Validation binary_cross_entropy = 0.403037
Epoch 263
Validation binary_cross_entropy = 0.392856
Epoch 264
Loss = 3.9824e-01, PNorm = 40.7921, GNorm = 3.2208, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.390323
Epoch 265
Validation binary_cross_entropy = 0.437946
Epoch 266
Validation binary_cross_entropy = 0.435497
Epoch 267
Validation binary_cross_entropy = 0.438631
Epoch 268
Validation binary_cross_entropy = 0.436616
Epoch 269
Loss = 4.1172e-01, PNorm = 40.8281, GNorm = 4.1940, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.423159
Epoch 270
Validation binary_cross_entropy = 0.402606
Epoch 271
Validation binary_cross_entropy = 0.398348
Epoch 272
Validation binary_cross_entropy = 0.404607
Epoch 273
Validation binary_cross_entropy = 0.414617
Epoch 274
Loss = 2.9572e-01, PNorm = 40.8617, GNorm = 2.3412, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.448492
Epoch 275
Validation binary_cross_entropy = 0.461400
Epoch 276
Validation binary_cross_entropy = 0.448659
Epoch 277
Validation binary_cross_entropy = 0.411338
Epoch 278
Validation binary_cross_entropy = 0.382807
Epoch 279
Loss = 3.5063e-01, PNorm = 40.8988, GNorm = 3.9713, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.369009
Epoch 280
Validation binary_cross_entropy = 0.367874
Epoch 281
Validation binary_cross_entropy = 0.375249
Epoch 282
Validation binary_cross_entropy = 0.384581
Epoch 283
Validation binary_cross_entropy = 0.413148
Epoch 284
Loss = 3.1112e-01, PNorm = 40.9328, GNorm = 2.5118, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.452055
Epoch 285
Validation binary_cross_entropy = 0.462368
Epoch 286
Validation binary_cross_entropy = 0.426987
Epoch 287
Validation binary_cross_entropy = 0.399567
Epoch 288
Validation binary_cross_entropy = 0.390030
Epoch 289
Loss = 2.3701e-01, PNorm = 40.9657, GNorm = 3.0179, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.375872
Epoch 290
Validation binary_cross_entropy = 0.365865
Epoch 291
Validation binary_cross_entropy = 0.374489
Epoch 292
Validation binary_cross_entropy = 0.376639
Epoch 293
Validation binary_cross_entropy = 0.372382
Epoch 294
Loss = 2.9838e-01, PNorm = 40.9980, GNorm = 1.9494, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.370878
Epoch 295
Validation binary_cross_entropy = 0.369516
Epoch 296
Validation binary_cross_entropy = 0.362953
Epoch 297
Validation binary_cross_entropy = 0.372440
Epoch 298
Validation binary_cross_entropy = 0.388053
Epoch 299
Loss = 2.9240e-01, PNorm = 41.0279, GNorm = 2.5307, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.394620
Model 0 best validation binary_cross_entropy = 0.277299 on epoch 18
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.195285
Ensemble test binary_cross_entropy = 0.195285
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 1,303,001
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.325067
Epoch 1
Validation binary_cross_entropy = 0.444939
Epoch 2
Validation binary_cross_entropy = 0.931544
Epoch 3
Validation binary_cross_entropy = 0.388926
Epoch 4
Loss = 9.5940e-01, PNorm = 38.3090, GNorm = 11.9409, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.383937
Epoch 5
Validation binary_cross_entropy = 1.237411
Epoch 6
Validation binary_cross_entropy = 0.363581
Epoch 7
Validation binary_cross_entropy = 0.289615
Epoch 8
Validation binary_cross_entropy = 0.966672
Epoch 9
Loss = 9.7827e-01, PNorm = 38.3912, GNorm = 13.2891, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.380937
Epoch 10
Validation binary_cross_entropy = 0.362072
Epoch 11
Validation binary_cross_entropy = 0.443699
Epoch 12
Validation binary_cross_entropy = 0.403718
Epoch 13
Validation binary_cross_entropy = 0.544784
Epoch 14
Loss = 8.9562e-01, PNorm = 38.5017, GNorm = 7.1008, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.443155
Epoch 15
Validation binary_cross_entropy = 0.381302
Epoch 16
Validation binary_cross_entropy = 0.535427
Epoch 17
Validation binary_cross_entropy = 0.391957
Epoch 18
Validation binary_cross_entropy = 0.404507
Epoch 19
Loss = 5.4121e-01, PNorm = 38.6062, GNorm = 5.2136, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.567433
Epoch 20
Validation binary_cross_entropy = 0.306349
Epoch 21
Validation binary_cross_entropy = 0.324029
Epoch 22
Validation binary_cross_entropy = 0.824425
Epoch 23
Validation binary_cross_entropy = 0.522741
Epoch 24
Loss = 7.3131e-01, PNorm = 38.6907, GNorm = 8.7472, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.310706
Epoch 25
Validation binary_cross_entropy = 0.451831
Epoch 26
Validation binary_cross_entropy = 0.715852
Epoch 27
Validation binary_cross_entropy = 0.796482
Epoch 28
Validation binary_cross_entropy = 0.350063
Epoch 29
Loss = 7.1892e-01, PNorm = 38.7634, GNorm = 7.5104, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.299173
Epoch 30
Validation binary_cross_entropy = 0.365757
Epoch 31
Validation binary_cross_entropy = 0.469290
Epoch 32
Validation binary_cross_entropy = 0.579226
Epoch 33
Validation binary_cross_entropy = 0.367394
Epoch 34
Loss = 6.1689e-01, PNorm = 38.8279, GNorm = 6.6591, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.390572
Epoch 35
Validation binary_cross_entropy = 0.737150
Epoch 36
Validation binary_cross_entropy = 0.894499
Epoch 37
Validation binary_cross_entropy = 0.566369
Epoch 38
Validation binary_cross_entropy = 0.350364
Epoch 39
Loss = 5.9588e-01, PNorm = 38.8838, GNorm = 4.5186, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.318530
Epoch 40
Validation binary_cross_entropy = 0.399013
Epoch 41
Validation binary_cross_entropy = 0.628466
Epoch 42
Validation binary_cross_entropy = 0.619493
Epoch 43
Validation binary_cross_entropy = 0.408226
Epoch 44
Loss = 5.6562e-01, PNorm = 38.9366, GNorm = 4.4950, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.316962
Epoch 45
Validation binary_cross_entropy = 0.353391
Epoch 46
Validation binary_cross_entropy = 0.563115
Epoch 47
Validation binary_cross_entropy = 0.725829
Epoch 48
Validation binary_cross_entropy = 0.511601
Epoch 49
Loss = 6.5483e-01, PNorm = 38.9874, GNorm = 4.9790, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.393094
Epoch 50
Validation binary_cross_entropy = 0.400467
Epoch 51
Validation binary_cross_entropy = 0.443829
Epoch 52
Validation binary_cross_entropy = 0.529905
Epoch 53
Validation binary_cross_entropy = 0.576784
Epoch 54
Loss = 5.3400e-01, PNorm = 39.0387, GNorm = 4.7183, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.549375
Epoch 55
Validation binary_cross_entropy = 0.468614
Epoch 56
Validation binary_cross_entropy = 0.442861
Epoch 57
Validation binary_cross_entropy = 0.493765
Epoch 58
Validation binary_cross_entropy = 0.588878
Epoch 59
Loss = 5.5557e-01, PNorm = 39.0869, GNorm = 3.7281, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.546384
Epoch 60
Validation binary_cross_entropy = 0.506576
Epoch 61
Validation binary_cross_entropy = 0.491771
Epoch 62
Validation binary_cross_entropy = 0.493628
Epoch 63
Validation binary_cross_entropy = 0.489793
Epoch 64
Loss = 4.3286e-01, PNorm = 39.1372, GNorm = 3.0143, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.514001
Epoch 65
Validation binary_cross_entropy = 0.524424
Epoch 66
Validation binary_cross_entropy = 0.550948
Epoch 67
Validation binary_cross_entropy = 0.446731
Epoch 68
Validation binary_cross_entropy = 0.382883
Epoch 69
Loss = 4.6842e-01, PNorm = 39.1812, GNorm = 3.0561, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.386894
Epoch 70
Validation binary_cross_entropy = 0.423283
Epoch 71
Validation binary_cross_entropy = 0.475809
Epoch 72
Validation binary_cross_entropy = 0.534290
Epoch 73
Validation binary_cross_entropy = 0.488629
Epoch 74
Loss = 4.9334e-01, PNorm = 39.2241, GNorm = 3.0739, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.477086
Epoch 75
Validation binary_cross_entropy = 0.457897
Epoch 76
Validation binary_cross_entropy = 0.475686
Epoch 77
Validation binary_cross_entropy = 0.461568
Epoch 78
Validation binary_cross_entropy = 0.462379
Epoch 79
Loss = 3.9195e-01, PNorm = 39.2680, GNorm = 2.6156, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.460745
Epoch 80
Validation binary_cross_entropy = 0.431825
Epoch 81
Validation binary_cross_entropy = 0.439251
Epoch 82
Validation binary_cross_entropy = 0.471445
Epoch 83
Validation binary_cross_entropy = 0.488640
Epoch 84
Loss = 4.6741e-01, PNorm = 39.3109, GNorm = 3.2822, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.442383
Epoch 85
Validation binary_cross_entropy = 0.377106
Epoch 86
Validation binary_cross_entropy = 0.373428
Epoch 87
Validation binary_cross_entropy = 0.406899
Epoch 88
Validation binary_cross_entropy = 0.473585
Epoch 89
Loss = 4.4923e-01, PNorm = 39.3525, GNorm = 2.5129, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.540417
Epoch 90
Validation binary_cross_entropy = 0.564155
Epoch 91
Validation binary_cross_entropy = 0.517186
Epoch 92
Validation binary_cross_entropy = 0.511295
Epoch 93
Validation binary_cross_entropy = 0.543351
Epoch 94
Loss = 4.7403e-01, PNorm = 39.3975, GNorm = 2.7786, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.507377
Epoch 95
Validation binary_cross_entropy = 0.417021
Epoch 96
Validation binary_cross_entropy = 0.373898
Epoch 97
Validation binary_cross_entropy = 0.422428
Epoch 98
Validation binary_cross_entropy = 0.534345
Epoch 99
Loss = 4.1717e-01, PNorm = 39.4440, GNorm = 3.4300, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.512445
Epoch 100
Validation binary_cross_entropy = 0.451325
Epoch 101
Validation binary_cross_entropy = 0.399961
Epoch 102
Validation binary_cross_entropy = 0.403802
Epoch 103
Validation binary_cross_entropy = 0.473051
Epoch 104
Loss = 4.2603e-01, PNorm = 39.4909, GNorm = 2.3042, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.600436
Epoch 105
Validation binary_cross_entropy = 0.579526
Epoch 106
Validation binary_cross_entropy = 0.492204
Epoch 107
Validation binary_cross_entropy = 0.452981
Epoch 108
Validation binary_cross_entropy = 0.499889
Epoch 109
Loss = 4.9667e-01, PNorm = 39.5378, GNorm = 2.9654, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.664230
Epoch 110
Validation binary_cross_entropy = 0.728710
Epoch 111
Validation binary_cross_entropy = 0.680969
Epoch 112
Validation binary_cross_entropy = 0.456179
Epoch 113
Validation binary_cross_entropy = 0.426524
Epoch 114
Loss = 4.0038e-01, PNorm = 39.5829, GNorm = 2.8183, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.428776
Epoch 115
Validation binary_cross_entropy = 0.457820
Epoch 116
Validation binary_cross_entropy = 0.497660
Epoch 117
Validation binary_cross_entropy = 0.458076
Epoch 118
Validation binary_cross_entropy = 0.439878
Epoch 119
Loss = 4.3127e-01, PNorm = 39.6254, GNorm = 4.3303, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.428516
Epoch 120
Validation binary_cross_entropy = 0.459141
Epoch 121
Validation binary_cross_entropy = 0.485629
Epoch 122
Validation binary_cross_entropy = 0.475402
Epoch 123
Validation binary_cross_entropy = 0.441006
Epoch 124
Loss = 4.3665e-01, PNorm = 39.6699, GNorm = 2.7234, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.400252
Epoch 125
Validation binary_cross_entropy = 0.389977
Epoch 126
Validation binary_cross_entropy = 0.411381
Epoch 127
Validation binary_cross_entropy = 0.417118
Epoch 128
Validation binary_cross_entropy = 0.419763
Epoch 129
Loss = 4.1580e-01, PNorm = 39.7155, GNorm = 2.8413, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.405693
Epoch 130
Validation binary_cross_entropy = 0.404192
Epoch 131
Validation binary_cross_entropy = 0.413136
Epoch 132
Validation binary_cross_entropy = 0.387294
Epoch 133
Validation binary_cross_entropy = 0.398261
Epoch 134
Loss = 4.7360e-01, PNorm = 39.7619, GNorm = 3.6992, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.398454
Epoch 135
Validation binary_cross_entropy = 0.435499
Epoch 136
Validation binary_cross_entropy = 0.433012
Epoch 137
Validation binary_cross_entropy = 0.394416
Epoch 138
Validation binary_cross_entropy = 0.347091
Epoch 139
Loss = 4.5280e-01, PNorm = 39.8133, GNorm = 2.8433, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.335052
Epoch 140
Validation binary_cross_entropy = 0.381570
Epoch 141
Validation binary_cross_entropy = 0.542044
Epoch 142
Validation binary_cross_entropy = 0.577920
Epoch 143
Validation binary_cross_entropy = 0.455954
Epoch 144
Loss = 6.0684e-01, PNorm = 39.8614, GNorm = 5.0635, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.356906
Epoch 145
Validation binary_cross_entropy = 0.341436
Epoch 146
Validation binary_cross_entropy = 0.398690
Epoch 147
Validation binary_cross_entropy = 0.445875
Epoch 148
Validation binary_cross_entropy = 0.463596
Epoch 149
Loss = 4.0000e-01, PNorm = 39.9114, GNorm = 3.1478, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.460310
Epoch 150
Validation binary_cross_entropy = 0.417992
Epoch 151
Validation binary_cross_entropy = 0.393167
Epoch 152
Validation binary_cross_entropy = 0.409470
Epoch 153
Validation binary_cross_entropy = 0.414006
Epoch 154
Loss = 4.8618e-01, PNorm = 39.9592, GNorm = 4.7832, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.370465
Epoch 155
Validation binary_cross_entropy = 0.333270
Epoch 156
Validation binary_cross_entropy = 0.343239
Epoch 157
Validation binary_cross_entropy = 0.388891
Epoch 158
Validation binary_cross_entropy = 0.433794
Epoch 159
Loss = 3.9843e-01, PNorm = 40.0076, GNorm = 2.8388, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.394447
Epoch 160
Validation binary_cross_entropy = 0.347418
Epoch 161
Validation binary_cross_entropy = 0.329305
Epoch 162
Validation binary_cross_entropy = 0.342565
Epoch 163
Validation binary_cross_entropy = 0.375416
Epoch 164
Loss = 3.3860e-01, PNorm = 40.0529, GNorm = 2.2420, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.443641
Epoch 165
Validation binary_cross_entropy = 0.525202
Epoch 166
Validation binary_cross_entropy = 0.459672
Epoch 167
Validation binary_cross_entropy = 0.371321
Epoch 168
Validation binary_cross_entropy = 0.323387
Epoch 169
Loss = 4.1139e-01, PNorm = 40.0998, GNorm = 2.9826, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.325491
Epoch 170
Validation binary_cross_entropy = 0.365526
Epoch 171
Validation binary_cross_entropy = 0.406385
Epoch 172
Validation binary_cross_entropy = 0.397850
Epoch 173
Validation binary_cross_entropy = 0.388714
Epoch 174
Loss = 3.9061e-01, PNorm = 40.1420, GNorm = 3.2035, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.374430
Epoch 175
Validation binary_cross_entropy = 0.369849
Epoch 176
Validation binary_cross_entropy = 0.381520
Epoch 177
Validation binary_cross_entropy = 0.435320
Epoch 178
Validation binary_cross_entropy = 0.445107
Epoch 179
Loss = 4.0959e-01, PNorm = 40.1817, GNorm = 3.4409, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.397460
Epoch 180
Validation binary_cross_entropy = 0.389443
Epoch 181
Validation binary_cross_entropy = 0.392546
Epoch 182
Validation binary_cross_entropy = 0.386260
Epoch 183
Validation binary_cross_entropy = 0.378494
Epoch 184
Loss = 3.8360e-01, PNorm = 40.2246, GNorm = 3.5966, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.383653
Epoch 185
Validation binary_cross_entropy = 0.364122
Epoch 186
Validation binary_cross_entropy = 0.363444
Epoch 187
Validation binary_cross_entropy = 0.400443
Epoch 188
Validation binary_cross_entropy = 0.423121
Epoch 189
Loss = 3.4352e-01, PNorm = 40.2692, GNorm = 3.0871, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.399452
Epoch 190
Validation binary_cross_entropy = 0.360628
Epoch 191
Validation binary_cross_entropy = 0.360846
Epoch 192
Validation binary_cross_entropy = 0.367551
Epoch 193
Validation binary_cross_entropy = 0.366214
Epoch 194
Loss = 3.2898e-01, PNorm = 40.3117, GNorm = 3.0761, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.376854
Epoch 195
Validation binary_cross_entropy = 0.397684
Epoch 196
Validation binary_cross_entropy = 0.419863
Epoch 197
Validation binary_cross_entropy = 0.431426
Epoch 198
Validation binary_cross_entropy = 0.399538
Epoch 199
Loss = 3.9767e-01, PNorm = 40.3519, GNorm = 3.4396, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.409256
Epoch 200
Validation binary_cross_entropy = 0.374200
Epoch 201
Validation binary_cross_entropy = 0.356786
Epoch 202
Validation binary_cross_entropy = 0.351844
Epoch 203
Validation binary_cross_entropy = 0.366685
Epoch 204
Loss = 3.1936e-01, PNorm = 40.3932, GNorm = 2.0770, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.382197
Epoch 205
Validation binary_cross_entropy = 0.401073
Epoch 206
Validation binary_cross_entropy = 0.402205
Epoch 207
Validation binary_cross_entropy = 0.385451
Epoch 208
Validation binary_cross_entropy = 0.358808
Epoch 209
Loss = 3.0945e-01, PNorm = 40.4345, GNorm = 3.4827, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.362725
Epoch 210
Validation binary_cross_entropy = 0.396676
Epoch 211
Validation binary_cross_entropy = 0.422988
Epoch 212
Validation binary_cross_entropy = 0.439811
Epoch 213
Validation binary_cross_entropy = 0.422760
Epoch 214
Loss = 4.0268e-01, PNorm = 40.4740, GNorm = 2.9825, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.426205
Epoch 215
Validation binary_cross_entropy = 0.403186
Epoch 216
Validation binary_cross_entropy = 0.396386
Epoch 217
Validation binary_cross_entropy = 0.433545
Epoch 218
Validation binary_cross_entropy = 0.510884
Epoch 219
Loss = 3.7424e-01, PNorm = 40.5099, GNorm = 2.2220, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.537951
Epoch 220
Validation binary_cross_entropy = 0.541967
Epoch 221
Validation binary_cross_entropy = 0.484322
Epoch 222
Validation binary_cross_entropy = 0.415293
Epoch 223
Validation binary_cross_entropy = 0.372249
Epoch 224
Loss = 2.3977e-01, PNorm = 40.5482, GNorm = 2.3566, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.348993
Epoch 225
Validation binary_cross_entropy = 0.357178
Epoch 226
Validation binary_cross_entropy = 0.397125
Epoch 227
Validation binary_cross_entropy = 0.406502
Epoch 228
Validation binary_cross_entropy = 0.381250
Epoch 229
Loss = 2.7900e-01, PNorm = 40.5871, GNorm = 1.9675, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.358532
Epoch 230
Validation binary_cross_entropy = 0.364778
Epoch 231
Validation binary_cross_entropy = 0.403947
Epoch 232
Validation binary_cross_entropy = 0.443807
Epoch 233
Validation binary_cross_entropy = 0.461199
Epoch 234
Loss = 3.2789e-01, PNorm = 40.6238, GNorm = 2.5048, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.447545
Epoch 235
Validation binary_cross_entropy = 0.402563
Epoch 236
Validation binary_cross_entropy = 0.365098
Epoch 237
Validation binary_cross_entropy = 0.348849
Epoch 238
Validation binary_cross_entropy = 0.369168
Epoch 239
Loss = 2.7289e-01, PNorm = 40.6607, GNorm = 2.0575, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.390833
Epoch 240
Validation binary_cross_entropy = 0.411018
Epoch 241
Validation binary_cross_entropy = 0.437107
Epoch 242
Validation binary_cross_entropy = 0.420730
Epoch 243
Validation binary_cross_entropy = 0.399981
Epoch 244
Loss = 3.3967e-01, PNorm = 40.6968, GNorm = 2.1210, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.374053
Epoch 245
Validation binary_cross_entropy = 0.370103
Epoch 246
Validation binary_cross_entropy = 0.393237
Epoch 247
Validation binary_cross_entropy = 0.423769
Epoch 248
Validation binary_cross_entropy = 0.437091
Epoch 249
Loss = 3.2811e-01, PNorm = 40.7282, GNorm = 2.5967, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.429802
Epoch 250
Validation binary_cross_entropy = 0.427190
Epoch 251
Validation binary_cross_entropy = 0.416620
Epoch 252
Validation binary_cross_entropy = 0.407236
Epoch 253
Validation binary_cross_entropy = 0.407447
Epoch 254
Loss = 2.3684e-01, PNorm = 40.7614, GNorm = 2.3951, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.416686
Epoch 255
Validation binary_cross_entropy = 0.441544
Epoch 256
Validation binary_cross_entropy = 0.469454
Epoch 257
Validation binary_cross_entropy = 0.449752
Epoch 258
Validation binary_cross_entropy = 0.412908
Epoch 259
Loss = 2.7201e-01, PNorm = 40.7928, GNorm = 2.2177, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.395268
Epoch 260
Validation binary_cross_entropy = 0.384858
Epoch 261
Validation binary_cross_entropy = 0.367937
Epoch 262
Validation binary_cross_entropy = 0.356850
Epoch 263
Validation binary_cross_entropy = 0.350754
Epoch 264
Loss = 3.3373e-01, PNorm = 40.8289, GNorm = 3.2651, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.359893
Epoch 265
Validation binary_cross_entropy = 0.384381
Epoch 266
Validation binary_cross_entropy = 0.411353
Epoch 267
Validation binary_cross_entropy = 0.404277
Epoch 268
Validation binary_cross_entropy = 0.370888
Epoch 269
Loss = 2.5176e-01, PNorm = 40.8696, GNorm = 2.2404, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.351094
Epoch 270
Validation binary_cross_entropy = 0.348657
Epoch 271
Validation binary_cross_entropy = 0.351814
Epoch 272
Validation binary_cross_entropy = 0.365627
Epoch 273
Validation binary_cross_entropy = 0.384270
Epoch 274
Loss = 3.1166e-01, PNorm = 40.9082, GNorm = 2.2463, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.420259
Epoch 275
Validation binary_cross_entropy = 0.406609
Epoch 276
Validation binary_cross_entropy = 0.387998
Epoch 277
Validation binary_cross_entropy = 0.375429
Epoch 278
Validation binary_cross_entropy = 0.373949
Epoch 279
Loss = 3.3804e-01, PNorm = 40.9425, GNorm = 4.1487, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.387051
Epoch 280
Validation binary_cross_entropy = 0.397916
Epoch 281
Validation binary_cross_entropy = 0.389509
Epoch 282
Validation binary_cross_entropy = 0.369298
Epoch 283
Validation binary_cross_entropy = 0.357597
Epoch 284
Loss = 2.4007e-01, PNorm = 40.9760, GNorm = 2.1335, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.352181
Epoch 285
Validation binary_cross_entropy = 0.340947
Epoch 286
Validation binary_cross_entropy = 0.340298
Epoch 287
Validation binary_cross_entropy = 0.348001
Epoch 288
Validation binary_cross_entropy = 0.365432
Epoch 289
Loss = 3.4735e-01, PNorm = 41.0106, GNorm = 2.5605, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.381613
Epoch 290
Validation binary_cross_entropy = 0.396231
Epoch 291
Validation binary_cross_entropy = 0.384097
Epoch 292
Validation binary_cross_entropy = 0.361351
Epoch 293
Validation binary_cross_entropy = 0.343694
Epoch 294
Loss = 2.1691e-01, PNorm = 41.0420, GNorm = 2.0785, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.337566
Epoch 295
Validation binary_cross_entropy = 0.335707
Epoch 296
Validation binary_cross_entropy = 0.349862
Epoch 297
Validation binary_cross_entropy = 0.368771
Epoch 298
Validation binary_cross_entropy = 0.363851
Epoch 299
Loss = 2.9296e-01, PNorm = 41.0727, GNorm = 3.0942, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.343158
Model 0 best validation binary_cross_entropy = 0.289615 on epoch 7
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.196908
Ensemble test binary_cross_entropy = 0.196908
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 1,303,001
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.303250
Epoch 1
Validation binary_cross_entropy = 0.466098
Epoch 2
Validation binary_cross_entropy = 0.744376
Epoch 3
Validation binary_cross_entropy = 0.391756
Epoch 4
Loss = 9.1334e-01, PNorm = 38.3081, GNorm = 8.2575, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.311796
Epoch 5
Validation binary_cross_entropy = 0.630832
Epoch 6
Validation binary_cross_entropy = 0.539434
Epoch 7
Validation binary_cross_entropy = 0.269651
Epoch 8
Validation binary_cross_entropy = 0.311082
Epoch 9
Loss = 9.4312e-01, PNorm = 38.3919, GNorm = 10.0007, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.437120
Epoch 10
Validation binary_cross_entropy = 0.359575
Epoch 11
Validation binary_cross_entropy = 0.454432
Epoch 12
Validation binary_cross_entropy = 0.462892
Epoch 13
Validation binary_cross_entropy = 0.401820
Epoch 14
Loss = 7.4761e-01, PNorm = 38.5075, GNorm = 7.1928, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.444050
Epoch 15
Validation binary_cross_entropy = 1.133297
Epoch 16
Validation binary_cross_entropy = 0.467751
Epoch 17
Validation binary_cross_entropy = 0.287282
Epoch 18
Validation binary_cross_entropy = 0.319329
Epoch 19
Loss = 6.9104e-01, PNorm = 38.6108, GNorm = 5.4103, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.630684
Epoch 20
Validation binary_cross_entropy = 0.437335
Epoch 21
Validation binary_cross_entropy = 0.347867
Epoch 22
Validation binary_cross_entropy = 0.335210
Epoch 23
Validation binary_cross_entropy = 0.343001
Epoch 24
Loss = 6.5552e-01, PNorm = 38.7003, GNorm = 3.8019, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.419271
Epoch 25
Validation binary_cross_entropy = 0.370378
Epoch 26
Validation binary_cross_entropy = 0.311765
Epoch 27
Validation binary_cross_entropy = 0.330888
Epoch 28
Validation binary_cross_entropy = 0.410611
Epoch 29
Loss = 6.9870e-01, PNorm = 38.7762, GNorm = 7.7063, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.374051
Epoch 30
Validation binary_cross_entropy = 0.317428
Epoch 31
Validation binary_cross_entropy = 0.374046
Epoch 32
Validation binary_cross_entropy = 0.662657
Epoch 33
Validation binary_cross_entropy = 0.498389
Epoch 34
Loss = 6.6489e-01, PNorm = 38.8407, GNorm = 4.9128, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.332858
Epoch 35
Validation binary_cross_entropy = 0.422596
Epoch 36
Validation binary_cross_entropy = 0.718575
Epoch 37
Validation binary_cross_entropy = 0.498280
Epoch 38
Validation binary_cross_entropy = 0.371901
Epoch 39
Loss = 5.9513e-01, PNorm = 38.8973, GNorm = 5.0173, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.426272
Epoch 40
Validation binary_cross_entropy = 0.646914
Epoch 41
Validation binary_cross_entropy = 0.623875
Epoch 42
Validation binary_cross_entropy = 0.493150
Epoch 43
Validation binary_cross_entropy = 0.471993
Epoch 44
Loss = 5.4021e-01, PNorm = 38.9485, GNorm = 3.9452, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.407455
Epoch 45
Validation binary_cross_entropy = 0.459575
Epoch 46
Validation binary_cross_entropy = 0.647924
Epoch 47
Validation binary_cross_entropy = 0.720349
Epoch 48
Validation binary_cross_entropy = 0.556434
Epoch 49
Loss = 5.7705e-01, PNorm = 38.9968, GNorm = 3.3463, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.477465
Epoch 50
Validation binary_cross_entropy = 0.525834
Epoch 51
Validation binary_cross_entropy = 0.631681
Epoch 52
Validation binary_cross_entropy = 0.748345
Epoch 53
Validation binary_cross_entropy = 0.598095
Epoch 54
Loss = 5.0571e-01, PNorm = 39.0423, GNorm = 3.0953, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.493996
Epoch 55
Validation binary_cross_entropy = 0.506962
Epoch 56
Validation binary_cross_entropy = 0.660052
Epoch 57
Validation binary_cross_entropy = 0.778296
Epoch 58
Validation binary_cross_entropy = 0.722486
Epoch 59
Loss = 5.9955e-01, PNorm = 39.0799, GNorm = 3.6754, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.620150
Epoch 60
Validation binary_cross_entropy = 0.668664
Epoch 61
Validation binary_cross_entropy = 0.675307
Epoch 62
Validation binary_cross_entropy = 0.764897
Epoch 63
Validation binary_cross_entropy = 0.853607
Epoch 64
Loss = 4.6078e-01, PNorm = 39.1201, GNorm = 3.2797, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.892183
Epoch 65
Validation binary_cross_entropy = 0.731391
Epoch 66
Validation binary_cross_entropy = 0.518432
Epoch 67
Validation binary_cross_entropy = 0.430033
Epoch 68
Validation binary_cross_entropy = 0.415095
Epoch 69
Loss = 4.7860e-01, PNorm = 39.1629, GNorm = 3.4560, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.573781
Epoch 70
Validation binary_cross_entropy = 0.675196
Epoch 71
Validation binary_cross_entropy = 0.734325
Epoch 72
Validation binary_cross_entropy = 0.546025
Epoch 73
Validation binary_cross_entropy = 0.453006
Epoch 74
Loss = 5.0533e-01, PNorm = 39.2038, GNorm = 3.9811, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.455763
Epoch 75
Validation binary_cross_entropy = 0.509543
Epoch 76
Validation binary_cross_entropy = 0.548097
Epoch 77
Validation binary_cross_entropy = 0.453540
Epoch 78
Validation binary_cross_entropy = 0.400029
Epoch 79
Loss = 4.6778e-01, PNorm = 39.2456, GNorm = 3.4011, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.422833
Epoch 80
Validation binary_cross_entropy = 0.523870
Epoch 81
Validation binary_cross_entropy = 0.635386
Epoch 82
Validation binary_cross_entropy = 0.500376
Epoch 83
Validation binary_cross_entropy = 0.400650
Epoch 84
Loss = 5.7969e-01, PNorm = 39.2890, GNorm = 3.2346, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.366611
Epoch 85
Validation binary_cross_entropy = 0.384172
Epoch 86
Validation binary_cross_entropy = 0.445823
Epoch 87
Validation binary_cross_entropy = 0.521669
Epoch 88
Validation binary_cross_entropy = 0.555959
Epoch 89
Loss = 3.9551e-01, PNorm = 39.3361, GNorm = 3.6137, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.539337
Epoch 90
Validation binary_cross_entropy = 0.460292
Epoch 91
Validation binary_cross_entropy = 0.472836
Epoch 92
Validation binary_cross_entropy = 0.650746
Epoch 93
Validation binary_cross_entropy = 0.944287
Epoch 94
Loss = 4.4140e-01, PNorm = 39.3850, GNorm = 3.2017, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 1.083994
Epoch 95
Validation binary_cross_entropy = 1.007597
Epoch 96
Validation binary_cross_entropy = 0.819228
Epoch 97
Validation binary_cross_entropy = 0.578697
Epoch 98
Validation binary_cross_entropy = 0.697781
Epoch 99
Loss = 3.8161e-01, PNorm = 39.4310, GNorm = 2.4864, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.818831
Epoch 100
Validation binary_cross_entropy = 0.889505
Epoch 101
Validation binary_cross_entropy = 0.725742
Epoch 102
Validation binary_cross_entropy = 0.566161
Epoch 103
Validation binary_cross_entropy = 0.597784
Epoch 104
Loss = 4.4572e-01, PNorm = 39.4740, GNorm = 2.9196, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.718552
Epoch 105
Validation binary_cross_entropy = 0.573398
Epoch 106
Validation binary_cross_entropy = 0.432350
Epoch 107
Validation binary_cross_entropy = 0.381662
Epoch 108
Validation binary_cross_entropy = 0.413774
Epoch 109
Loss = 3.8231e-01, PNorm = 39.5167, GNorm = 2.2149, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.538684
Epoch 110
Validation binary_cross_entropy = 0.695766
Epoch 111
Validation binary_cross_entropy = 0.687817
Epoch 112
Validation binary_cross_entropy = 0.541282
Epoch 113
Validation binary_cross_entropy = 0.394273
Epoch 114
Loss = 4.8748e-01, PNorm = 39.5630, GNorm = 3.1716, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.378929
Epoch 115
Validation binary_cross_entropy = 0.473249
Epoch 116
Validation binary_cross_entropy = 0.574437
Epoch 117
Validation binary_cross_entropy = 0.542674
Epoch 118
Validation binary_cross_entropy = 0.448543
Epoch 119
Loss = 3.8957e-01, PNorm = 39.6080, GNorm = 2.3232, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.419835
Epoch 120
Validation binary_cross_entropy = 0.441207
Epoch 121
Validation binary_cross_entropy = 0.468402
Epoch 122
Validation binary_cross_entropy = 0.516955
Epoch 123
Validation binary_cross_entropy = 0.549355
Epoch 124
Loss = 4.4910e-01, PNorm = 39.6523, GNorm = 2.7412, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.563400
Epoch 125
Validation binary_cross_entropy = 0.555767
Epoch 126
Validation binary_cross_entropy = 0.493188
Epoch 127
Validation binary_cross_entropy = 0.435632
Epoch 128
Validation binary_cross_entropy = 0.405666
Epoch 129
Loss = 3.5754e-01, PNorm = 39.6943, GNorm = 3.3336, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.432371
Epoch 130
Validation binary_cross_entropy = 0.493865
Epoch 131
Validation binary_cross_entropy = 0.444317
Epoch 132
Validation binary_cross_entropy = 0.407006
Epoch 133
Validation binary_cross_entropy = 0.408008
Epoch 134
Loss = 4.2413e-01, PNorm = 39.7339, GNorm = 3.4518, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.385001
Epoch 135
Validation binary_cross_entropy = 0.368024
Epoch 136
Validation binary_cross_entropy = 0.404908
Epoch 137
Validation binary_cross_entropy = 0.438126
Epoch 138
Validation binary_cross_entropy = 0.481959
Epoch 139
Loss = 4.2094e-01, PNorm = 39.7745, GNorm = 3.1125, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.469407
Epoch 140
Validation binary_cross_entropy = 0.442550
Epoch 141
Validation binary_cross_entropy = 0.446138
Epoch 142
Validation binary_cross_entropy = 0.449738
Epoch 143
Validation binary_cross_entropy = 0.447341
Epoch 144
Loss = 5.1179e-01, PNorm = 39.8147, GNorm = 4.7368, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.430798
Epoch 145
Validation binary_cross_entropy = 0.456723
Epoch 146
Validation binary_cross_entropy = 0.536644
Epoch 147
Validation binary_cross_entropy = 0.503734
Epoch 148
Validation binary_cross_entropy = 0.473183
Epoch 149
Loss = 4.0959e-01, PNorm = 39.8578, GNorm = 1.8701, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.508754
Epoch 150
Validation binary_cross_entropy = 0.522994
Epoch 151
Validation binary_cross_entropy = 0.550618
Epoch 152
Validation binary_cross_entropy = 0.453556
Epoch 153
Validation binary_cross_entropy = 0.415190
Epoch 154
Loss = 4.4712e-01, PNorm = 39.9009, GNorm = 3.0812, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.425529
Epoch 155
Validation binary_cross_entropy = 0.457638
Epoch 156
Validation binary_cross_entropy = 0.529610
Epoch 157
Validation binary_cross_entropy = 0.509058
Epoch 158
Validation binary_cross_entropy = 0.374705
Epoch 159
Loss = 4.3568e-01, PNorm = 39.9427, GNorm = 4.0733, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.309204
Epoch 160
Validation binary_cross_entropy = 0.315574
Epoch 161
Validation binary_cross_entropy = 0.362533
Epoch 162
Validation binary_cross_entropy = 0.446798
Epoch 163
Validation binary_cross_entropy = 0.531111
Epoch 164
Loss = 5.0082e-01, PNorm = 39.9846, GNorm = 3.1460, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.488228
Epoch 165
Validation binary_cross_entropy = 0.437448
Epoch 166
Validation binary_cross_entropy = 0.396991
Epoch 167
Validation binary_cross_entropy = 0.374279
Epoch 168
Validation binary_cross_entropy = 0.382094
Epoch 169
Loss = 3.9370e-01, PNorm = 40.0276, GNorm = 2.8383, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.388990
Epoch 170
Validation binary_cross_entropy = 0.400415
Epoch 171
Validation binary_cross_entropy = 0.406382
Epoch 172
Validation binary_cross_entropy = 0.405453
Epoch 173
Validation binary_cross_entropy = 0.414463
Epoch 174
Loss = 3.4457e-01, PNorm = 40.0701, GNorm = 3.0301, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.449662
Epoch 175
Validation binary_cross_entropy = 0.461136
Epoch 176
Validation binary_cross_entropy = 0.424308
Epoch 177
Validation binary_cross_entropy = 0.398725
Epoch 178
Validation binary_cross_entropy = 0.352230
Epoch 179
Loss = 3.8274e-01, PNorm = 40.1145, GNorm = 2.7485, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.335777
Epoch 180
Validation binary_cross_entropy = 0.375673
Epoch 181
Validation binary_cross_entropy = 0.464031
Epoch 182
Validation binary_cross_entropy = 0.520412
Epoch 183
Validation binary_cross_entropy = 0.551424
Epoch 184
Loss = 4.7207e-01, PNorm = 40.1562, GNorm = 3.0053, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.474148
Epoch 185
Validation binary_cross_entropy = 0.384527
Epoch 186
Validation binary_cross_entropy = 0.339559
Epoch 187
Validation binary_cross_entropy = 0.367763
Epoch 188
Validation binary_cross_entropy = 0.422326
Epoch 189
Loss = 3.1071e-01, PNorm = 40.1977, GNorm = 2.4123, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.470382
Epoch 190
Validation binary_cross_entropy = 0.471231
Epoch 191
Validation binary_cross_entropy = 0.430508
Epoch 192
Validation binary_cross_entropy = 0.414697
Epoch 193
Validation binary_cross_entropy = 0.430979
Epoch 194
Loss = 4.0667e-01, PNorm = 40.2383, GNorm = 2.5207, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.483421
Epoch 195
Validation binary_cross_entropy = 0.499444
Epoch 196
Validation binary_cross_entropy = 0.472806
Epoch 197
Validation binary_cross_entropy = 0.434186
Epoch 198
Validation binary_cross_entropy = 0.416217
Epoch 199
Loss = 4.1885e-01, PNorm = 40.2803, GNorm = 3.3299, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.405062
Epoch 200
Validation binary_cross_entropy = 0.381892
Epoch 201
Validation binary_cross_entropy = 0.389719
Epoch 202
Validation binary_cross_entropy = 0.423245
Epoch 203
Validation binary_cross_entropy = 0.477075
Epoch 204
Loss = 3.7213e-01, PNorm = 40.3182, GNorm = 3.4733, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.519508
Epoch 205
Validation binary_cross_entropy = 0.588389
Epoch 206
Validation binary_cross_entropy = 0.589266
Epoch 207
Validation binary_cross_entropy = 0.449441
Epoch 208
Validation binary_cross_entropy = 0.411263
Epoch 209
Loss = 3.7280e-01, PNorm = 40.3576, GNorm = 3.3368, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.471568
Epoch 210
Validation binary_cross_entropy = 0.524191
Epoch 211
Validation binary_cross_entropy = 0.498959
Epoch 212
Validation binary_cross_entropy = 0.478194
Epoch 213
Validation binary_cross_entropy = 0.425205
Epoch 214
Loss = 4.5771e-01, PNorm = 40.3998, GNorm = 2.6833, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.407477
Epoch 215
Validation binary_cross_entropy = 0.379304
Epoch 216
Validation binary_cross_entropy = 0.379748
Epoch 217
Validation binary_cross_entropy = 0.402028
Epoch 218
Validation binary_cross_entropy = 0.513657
Epoch 219
Loss = 2.8943e-01, PNorm = 40.4379, GNorm = 1.8975, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.638086
Epoch 220
Validation binary_cross_entropy = 0.719768
Epoch 221
Validation binary_cross_entropy = 0.621616
Epoch 222
Validation binary_cross_entropy = 0.468120
Epoch 223
Validation binary_cross_entropy = 0.379542
Epoch 224
Loss = 2.5201e-01, PNorm = 40.4763, GNorm = 2.0620, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.349767
Epoch 225
Validation binary_cross_entropy = 0.341904
Epoch 226
Validation binary_cross_entropy = 0.348206
Epoch 227
Validation binary_cross_entropy = 0.370806
Epoch 228
Validation binary_cross_entropy = 0.390983
Epoch 229
Loss = 3.9441e-01, PNorm = 40.5132, GNorm = 3.7867, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.357492
Epoch 230
Validation binary_cross_entropy = 0.326629
Epoch 231
Validation binary_cross_entropy = 0.329101
Epoch 232
Validation binary_cross_entropy = 0.366625
Epoch 233
Validation binary_cross_entropy = 0.424245
Epoch 234
Loss = 3.5474e-01, PNorm = 40.5502, GNorm = 4.0622, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.440924
Epoch 235
Validation binary_cross_entropy = 0.393543
Epoch 236
Validation binary_cross_entropy = 0.366788
Epoch 237
Validation binary_cross_entropy = 0.369362
Epoch 238
Validation binary_cross_entropy = 0.382014
Epoch 239
Loss = 3.2891e-01, PNorm = 40.5868, GNorm = 3.0149, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.385272
Epoch 240
Validation binary_cross_entropy = 0.383674
Epoch 241
Validation binary_cross_entropy = 0.389462
Epoch 242
Validation binary_cross_entropy = 0.400812
Epoch 243
Validation binary_cross_entropy = 0.406972
Epoch 244
Loss = 3.3830e-01, PNorm = 40.6220, GNorm = 3.0158, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.398601
Epoch 245
Validation binary_cross_entropy = 0.379161
Epoch 246
Validation binary_cross_entropy = 0.368964
Epoch 247
Validation binary_cross_entropy = 0.378025
Epoch 248
Validation binary_cross_entropy = 0.387586
Epoch 249
Loss = 3.3224e-01, PNorm = 40.6522, GNorm = 2.3985, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.382020
Epoch 250
Validation binary_cross_entropy = 0.384996
Epoch 251
Validation binary_cross_entropy = 0.396064
Epoch 252
Validation binary_cross_entropy = 0.401741
Epoch 253
Validation binary_cross_entropy = 0.400347
Epoch 254
Loss = 2.6334e-01, PNorm = 40.6820, GNorm = 2.6459, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.392053
Epoch 255
Validation binary_cross_entropy = 0.394767
Epoch 256
Validation binary_cross_entropy = 0.406341
Epoch 257
Validation binary_cross_entropy = 0.398604
Epoch 258
Validation binary_cross_entropy = 0.366444
Epoch 259
Loss = 2.8525e-01, PNorm = 40.7138, GNorm = 2.5062, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.344850
Epoch 260
Validation binary_cross_entropy = 0.343945
Epoch 261
Validation binary_cross_entropy = 0.364563
Epoch 262
Validation binary_cross_entropy = 0.382756
Epoch 263
Validation binary_cross_entropy = 0.405007
Epoch 264
Loss = 3.8257e-01, PNorm = 40.7479, GNorm = 3.7389, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.398161
Epoch 265
Validation binary_cross_entropy = 0.379566
Epoch 266
Validation binary_cross_entropy = 0.376647
Epoch 267
Validation binary_cross_entropy = 0.376773
Epoch 268
Validation binary_cross_entropy = 0.383613
Epoch 269
Loss = 2.2202e-01, PNorm = 40.7798, GNorm = 1.9559, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.404637
Epoch 270
Validation binary_cross_entropy = 0.405531
Epoch 271
Validation binary_cross_entropy = 0.401387
Epoch 272
Validation binary_cross_entropy = 0.397007
Epoch 273
Validation binary_cross_entropy = 0.397746
Epoch 274
Loss = 3.1740e-01, PNorm = 40.8100, GNorm = 2.0915, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.401305
Epoch 275
Validation binary_cross_entropy = 0.433397
Epoch 276
Validation binary_cross_entropy = 0.425565
Epoch 277
Validation binary_cross_entropy = 0.408038
Epoch 278
Validation binary_cross_entropy = 0.377906
Epoch 279
Loss = 3.7434e-01, PNorm = 40.8416, GNorm = 1.9903, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.363309
Epoch 280
Validation binary_cross_entropy = 0.366661
Epoch 281
Validation binary_cross_entropy = 0.371170
Epoch 282
Validation binary_cross_entropy = 0.407425
Epoch 283
Validation binary_cross_entropy = 0.426334
Epoch 284
Loss = 2.9992e-01, PNorm = 40.8754, GNorm = 2.4124, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.408450
Epoch 285
Validation binary_cross_entropy = 0.392036
Epoch 286
Validation binary_cross_entropy = 0.371650
Epoch 287
Validation binary_cross_entropy = 0.374753
Epoch 288
Validation binary_cross_entropy = 0.417075
Epoch 289
Loss = 3.4821e-01, PNorm = 40.9083, GNorm = 2.8312, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.444698
Epoch 290
Validation binary_cross_entropy = 0.454865
Epoch 291
Validation binary_cross_entropy = 0.444434
Epoch 292
Validation binary_cross_entropy = 0.419957
Epoch 293
Validation binary_cross_entropy = 0.419338
Epoch 294
Loss = 3.0323e-01, PNorm = 40.9386, GNorm = 2.5210, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.412020
Epoch 295
Validation binary_cross_entropy = 0.379898
Epoch 296
Validation binary_cross_entropy = 0.371648
Epoch 297
Validation binary_cross_entropy = 0.359373
Epoch 298
Validation binary_cross_entropy = 0.354993
Epoch 299
Loss = 3.4284e-01, PNorm = 40.9740, GNorm = 3.8245, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.356673
Model 0 best validation binary_cross_entropy = 0.269651 on epoch 7
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.197607
Ensemble test binary_cross_entropy = 0.197607
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 1,303,001
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.307241
Epoch 1
Validation binary_cross_entropy = 0.488710
Epoch 2
Validation binary_cross_entropy = 0.649777
Epoch 3
Validation binary_cross_entropy = 0.368181
Epoch 4
Loss = 9.3473e-01, PNorm = 38.3089, GNorm = 9.0803, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.353942
Epoch 5
Validation binary_cross_entropy = 0.491021
Epoch 6
Validation binary_cross_entropy = 0.409325
Epoch 7
Validation binary_cross_entropy = 0.290803
Epoch 8
Validation binary_cross_entropy = 0.391037
Epoch 9
Loss = 8.8853e-01, PNorm = 38.3937, GNorm = 8.0892, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.437539
Epoch 10
Validation binary_cross_entropy = 0.342478
Epoch 11
Validation binary_cross_entropy = 0.295609
Epoch 12
Validation binary_cross_entropy = 0.478295
Epoch 13
Validation binary_cross_entropy = 0.736177
Epoch 14
Loss = 8.4761e-01, PNorm = 38.5120, GNorm = 5.8970, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.351783
Epoch 15
Validation binary_cross_entropy = 0.321555
Epoch 16
Validation binary_cross_entropy = 0.857044
Epoch 17
Validation binary_cross_entropy = 0.577343
Epoch 18
Validation binary_cross_entropy = 0.360684
Epoch 19
Loss = 6.9397e-01, PNorm = 38.6177, GNorm = 5.6374, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.659868
Epoch 20
Validation binary_cross_entropy = 0.461541
Epoch 21
Validation binary_cross_entropy = 0.408291
Epoch 22
Validation binary_cross_entropy = 0.371538
Epoch 23
Validation binary_cross_entropy = 0.431470
Epoch 24
Loss = 5.8613e-01, PNorm = 38.7046, GNorm = 5.2332, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.430226
Epoch 25
Validation binary_cross_entropy = 0.490650
Epoch 26
Validation binary_cross_entropy = 0.466211
Epoch 27
Validation binary_cross_entropy = 0.408631
Epoch 28
Validation binary_cross_entropy = 0.501971
Epoch 29
Loss = 7.4446e-01, PNorm = 38.7733, GNorm = 7.8422, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.623297
Epoch 30
Validation binary_cross_entropy = 0.417481
Epoch 31
Validation binary_cross_entropy = 0.360192
Epoch 32
Validation binary_cross_entropy = 0.598979
Epoch 33
Validation binary_cross_entropy = 1.019835
Epoch 34
Loss = 5.0863e-01, PNorm = 38.8358, GNorm = 4.3836, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.747991
Epoch 35
Validation binary_cross_entropy = 0.466611
Epoch 36
Validation binary_cross_entropy = 0.403938
Epoch 37
Validation binary_cross_entropy = 0.489663
Epoch 38
Validation binary_cross_entropy = 0.612678
Epoch 39
Loss = 6.4085e-01, PNorm = 38.8945, GNorm = 4.8691, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.471120
Epoch 40
Validation binary_cross_entropy = 0.380286
Epoch 41
Validation binary_cross_entropy = 0.421787
Epoch 42
Validation binary_cross_entropy = 0.748456
Epoch 43
Validation binary_cross_entropy = 0.962232
Epoch 44
Loss = 5.7778e-01, PNorm = 38.9491, GNorm = 3.4377, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.759714
Epoch 45
Validation binary_cross_entropy = 0.614306
Epoch 46
Validation binary_cross_entropy = 0.656326
Epoch 47
Validation binary_cross_entropy = 0.962428
Epoch 48
Validation binary_cross_entropy = 0.747272
Epoch 49
Loss = 6.7043e-01, PNorm = 38.9988, GNorm = 4.2827, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.484243
Epoch 50
Validation binary_cross_entropy = 0.467411
Epoch 51
Validation binary_cross_entropy = 0.537912
Epoch 52
Validation binary_cross_entropy = 0.552549
Epoch 53
Validation binary_cross_entropy = 0.489362
Epoch 54
Loss = 5.1899e-01, PNorm = 39.0445, GNorm = 2.9100, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.430457
Epoch 55
Validation binary_cross_entropy = 0.422526
Epoch 56
Validation binary_cross_entropy = 0.461296
Epoch 57
Validation binary_cross_entropy = 0.510489
Epoch 58
Validation binary_cross_entropy = 0.639437
Epoch 59
Loss = 5.8345e-01, PNorm = 39.0886, GNorm = 3.3084, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.678719
Epoch 60
Validation binary_cross_entropy = 0.581121
Epoch 61
Validation binary_cross_entropy = 0.516376
Epoch 62
Validation binary_cross_entropy = 0.446539
Epoch 63
Validation binary_cross_entropy = 0.482280
Epoch 64
Loss = 4.3714e-01, PNorm = 39.1344, GNorm = 2.4108, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.586548
Epoch 65
Validation binary_cross_entropy = 0.635163
Epoch 66
Validation binary_cross_entropy = 0.556112
Epoch 67
Validation binary_cross_entropy = 0.434299
Epoch 68
Validation binary_cross_entropy = 0.374920
Epoch 69
Loss = 4.8303e-01, PNorm = 39.1812, GNorm = 3.5035, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.418370
Epoch 70
Validation binary_cross_entropy = 0.540747
Epoch 71
Validation binary_cross_entropy = 0.610545
Epoch 72
Validation binary_cross_entropy = 0.485753
Epoch 73
Validation binary_cross_entropy = 0.463337
Epoch 74
Loss = 4.8506e-01, PNorm = 39.2277, GNorm = 3.5690, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.506023
Epoch 75
Validation binary_cross_entropy = 0.605662
Epoch 76
Validation binary_cross_entropy = 0.522791
Epoch 77
Validation binary_cross_entropy = 0.430661
Epoch 78
Validation binary_cross_entropy = 0.433250
Epoch 79
Loss = 4.5290e-01, PNorm = 39.2704, GNorm = 2.9492, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.407622
Epoch 80
Validation binary_cross_entropy = 0.459018
Epoch 81
Validation binary_cross_entropy = 0.489348
Epoch 82
Validation binary_cross_entropy = 0.422907
Epoch 83
Validation binary_cross_entropy = 0.389846
Epoch 84
Loss = 4.6550e-01, PNorm = 39.3167, GNorm = 3.5266, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.407769
Epoch 85
Validation binary_cross_entropy = 0.467592
Epoch 86
Validation binary_cross_entropy = 0.485760
Epoch 87
Validation binary_cross_entropy = 0.512746
Epoch 88
Validation binary_cross_entropy = 0.436186
Epoch 89
Loss = 4.9453e-01, PNorm = 39.3666, GNorm = 2.7793, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.383805
Epoch 90
Validation binary_cross_entropy = 0.401075
Epoch 91
Validation binary_cross_entropy = 0.452593
Epoch 92
Validation binary_cross_entropy = 0.529254
Epoch 93
Validation binary_cross_entropy = 0.537223
Epoch 94
Loss = 5.0375e-01, PNorm = 39.4152, GNorm = 2.8326, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.483509
Epoch 95
Validation binary_cross_entropy = 0.433859
Epoch 96
Validation binary_cross_entropy = 0.390249
Epoch 97
Validation binary_cross_entropy = 0.449871
Epoch 98
Validation binary_cross_entropy = 0.511338
Epoch 99
Loss = 4.5893e-01, PNorm = 39.4619, GNorm = 3.0666, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.417587
Epoch 100
Validation binary_cross_entropy = 0.351285
Epoch 101
Validation binary_cross_entropy = 0.330040
Epoch 102
Validation binary_cross_entropy = 0.392402
Epoch 103
Validation binary_cross_entropy = 0.559561
Epoch 104
Loss = 3.9614e-01, PNorm = 39.5082, GNorm = 3.7361, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.594438
Epoch 105
Validation binary_cross_entropy = 0.478915
Epoch 106
Validation binary_cross_entropy = 0.370442
Epoch 107
Validation binary_cross_entropy = 0.372575
Epoch 108
Validation binary_cross_entropy = 0.494140
Epoch 109
Loss = 4.2818e-01, PNorm = 39.5592, GNorm = 3.1806, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.804219
Epoch 110
Validation binary_cross_entropy = 0.762275
Epoch 111
Validation binary_cross_entropy = 0.583363
Epoch 112
Validation binary_cross_entropy = 0.496314
Epoch 113
Validation binary_cross_entropy = 0.526896
Epoch 114
Loss = 5.2581e-01, PNorm = 39.6055, GNorm = 2.7113, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.438048
Epoch 115
Validation binary_cross_entropy = 0.419428
Epoch 116
Validation binary_cross_entropy = 0.401380
Epoch 117
Validation binary_cross_entropy = 0.394514
Epoch 118
Validation binary_cross_entropy = 0.397451
Epoch 119
Loss = 4.8602e-01, PNorm = 39.6479, GNorm = 3.1413, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.409424
Epoch 120
Validation binary_cross_entropy = 0.421370
Epoch 121
Validation binary_cross_entropy = 0.448064
Epoch 122
Validation binary_cross_entropy = 0.412546
Epoch 123
Validation binary_cross_entropy = 0.409452
Epoch 124
Loss = 4.3729e-01, PNorm = 39.6887, GNorm = 3.8509, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.407314
Epoch 125
Validation binary_cross_entropy = 0.413210
Epoch 126
Validation binary_cross_entropy = 0.438343
Epoch 127
Validation binary_cross_entropy = 0.455363
Epoch 128
Validation binary_cross_entropy = 0.497173
Epoch 129
Loss = 3.6018e-01, PNorm = 39.7340, GNorm = 2.3459, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.552174
Epoch 130
Validation binary_cross_entropy = 0.521802
Epoch 131
Validation binary_cross_entropy = 0.466517
Epoch 132
Validation binary_cross_entropy = 0.411248
Epoch 133
Validation binary_cross_entropy = 0.409407
Epoch 134
Loss = 3.9698e-01, PNorm = 39.7772, GNorm = 4.1914, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.478499
Epoch 135
Validation binary_cross_entropy = 0.504649
Epoch 136
Validation binary_cross_entropy = 0.440191
Epoch 137
Validation binary_cross_entropy = 0.396037
Epoch 138
Validation binary_cross_entropy = 0.394332
Epoch 139
Loss = 3.4320e-01, PNorm = 39.8165, GNorm = 2.4244, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.443945
Epoch 140
Validation binary_cross_entropy = 0.474302
Epoch 141
Validation binary_cross_entropy = 0.494536
Epoch 142
Validation binary_cross_entropy = 0.460361
Epoch 143
Validation binary_cross_entropy = 0.409242
Epoch 144
Loss = 4.6561e-01, PNorm = 39.8571, GNorm = 3.7215, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.360776
Epoch 145
Validation binary_cross_entropy = 0.384699
Epoch 146
Validation binary_cross_entropy = 0.472238
Epoch 147
Validation binary_cross_entropy = 0.509202
Epoch 148
Validation binary_cross_entropy = 0.470831
Epoch 149
Loss = 3.8922e-01, PNorm = 39.8997, GNorm = 2.5551, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.423081
Epoch 150
Validation binary_cross_entropy = 0.390033
Epoch 151
Validation binary_cross_entropy = 0.375958
Epoch 152
Validation binary_cross_entropy = 0.396153
Epoch 153
Validation binary_cross_entropy = 0.392633
Epoch 154
Loss = 3.8105e-01, PNorm = 39.9422, GNorm = 2.9687, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.399424
Epoch 155
Validation binary_cross_entropy = 0.425754
Epoch 156
Validation binary_cross_entropy = 0.412411
Epoch 157
Validation binary_cross_entropy = 0.412554
Epoch 158
Validation binary_cross_entropy = 0.404618
Epoch 159
Loss = 3.8148e-01, PNorm = 39.9833, GNorm = 2.5180, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.372272
Epoch 160
Validation binary_cross_entropy = 0.352130
Epoch 161
Validation binary_cross_entropy = 0.357750
Epoch 162
Validation binary_cross_entropy = 0.419500
Epoch 163
Validation binary_cross_entropy = 0.459115
Epoch 164
Loss = 3.1212e-01, PNorm = 40.0243, GNorm = 2.3923, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.445769
Epoch 165
Validation binary_cross_entropy = 0.454434
Epoch 166
Validation binary_cross_entropy = 0.403199
Epoch 167
Validation binary_cross_entropy = 0.361481
Epoch 168
Validation binary_cross_entropy = 0.360067
Epoch 169
Loss = 3.6187e-01, PNorm = 40.0643, GNorm = 3.0716, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.404398
Epoch 170
Validation binary_cross_entropy = 0.472450
Epoch 171
Validation binary_cross_entropy = 0.496602
Epoch 172
Validation binary_cross_entropy = 0.412447
Epoch 173
Validation binary_cross_entropy = 0.360047
Epoch 174
Loss = 3.8105e-01, PNorm = 40.1031, GNorm = 3.9419, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.362138
Epoch 175
Validation binary_cross_entropy = 0.410791
Epoch 176
Validation binary_cross_entropy = 0.494526
Epoch 177
Validation binary_cross_entropy = 0.543347
Epoch 178
Validation binary_cross_entropy = 0.492840
Epoch 179
Loss = 4.4122e-01, PNorm = 40.1432, GNorm = 3.1156, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.381977
Epoch 180
Validation binary_cross_entropy = 0.321352
Epoch 181
Validation binary_cross_entropy = 0.309393
Epoch 182
Validation binary_cross_entropy = 0.326313
Epoch 183
Validation binary_cross_entropy = 0.397136
Epoch 184
Loss = 4.1084e-01, PNorm = 40.1830, GNorm = 2.8636, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.499453
Epoch 185
Validation binary_cross_entropy = 0.510579
Epoch 186
Validation binary_cross_entropy = 0.452597
Epoch 187
Validation binary_cross_entropy = 0.427962
Epoch 188
Validation binary_cross_entropy = 0.404296
Epoch 189
Loss = 3.8365e-01, PNorm = 40.2211, GNorm = 2.8923, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.395342
Epoch 190
Validation binary_cross_entropy = 0.420352
Epoch 191
Validation binary_cross_entropy = 0.480581
Epoch 192
Validation binary_cross_entropy = 0.523822
Epoch 193
Validation binary_cross_entropy = 0.522918
Epoch 194
Loss = 2.9088e-01, PNorm = 40.2574, GNorm = 2.5173, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.492049
Epoch 195
Validation binary_cross_entropy = 0.487152
Epoch 196
Validation binary_cross_entropy = 0.488541
Epoch 197
Validation binary_cross_entropy = 0.453673
Epoch 198
Validation binary_cross_entropy = 0.385683
Epoch 199
Loss = 4.5540e-01, PNorm = 40.2927, GNorm = 3.0786, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.354585
Epoch 200
Validation binary_cross_entropy = 0.336392
Epoch 201
Validation binary_cross_entropy = 0.333215
Epoch 202
Validation binary_cross_entropy = 0.356099
Epoch 203
Validation binary_cross_entropy = 0.405445
Epoch 204
Loss = 3.0397e-01, PNorm = 40.3272, GNorm = 2.4344, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.469386
Epoch 205
Validation binary_cross_entropy = 0.555012
Epoch 206
Validation binary_cross_entropy = 0.573949
Epoch 207
Validation binary_cross_entropy = 0.483159
Epoch 208
Validation binary_cross_entropy = 0.407576
Epoch 209
Loss = 2.6304e-01, PNorm = 40.3632, GNorm = 2.1441, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.370160
Epoch 210
Validation binary_cross_entropy = 0.375441
Epoch 211
Validation binary_cross_entropy = 0.384993
Epoch 212
Validation binary_cross_entropy = 0.426940
Epoch 213
Validation binary_cross_entropy = 0.425356
Epoch 214
Loss = 3.3910e-01, PNorm = 40.3993, GNorm = 3.2377, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.411724
Epoch 215
Validation binary_cross_entropy = 0.378894
Epoch 216
Validation binary_cross_entropy = 0.362301
Epoch 217
Validation binary_cross_entropy = 0.381436
Epoch 218
Validation binary_cross_entropy = 0.390624
Epoch 219
Loss = 3.6951e-01, PNorm = 40.4370, GNorm = 2.0603, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.395076
Epoch 220
Validation binary_cross_entropy = 0.397077
Epoch 221
Validation binary_cross_entropy = 0.396144
Epoch 222
Validation binary_cross_entropy = 0.389201
Epoch 223
Validation binary_cross_entropy = 0.366772
Epoch 224
Loss = 4.0072e-01, PNorm = 40.4733, GNorm = 3.8050, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.341218
Epoch 225
Validation binary_cross_entropy = 0.334892
Epoch 226
Validation binary_cross_entropy = 0.377270
Epoch 227
Validation binary_cross_entropy = 0.418943
Epoch 228
Validation binary_cross_entropy = 0.411316
Epoch 229
Loss = 3.0510e-01, PNorm = 40.5092, GNorm = 3.1607, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.372539
Epoch 230
Validation binary_cross_entropy = 0.329327
Epoch 231
Validation binary_cross_entropy = 0.326436
Epoch 232
Validation binary_cross_entropy = 0.348709
Epoch 233
Validation binary_cross_entropy = 0.385380
Epoch 234
Loss = 3.2242e-01, PNorm = 40.5480, GNorm = 2.2644, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.416202
Epoch 235
Validation binary_cross_entropy = 0.412335
Epoch 236
Validation binary_cross_entropy = 0.366496
Epoch 237
Validation binary_cross_entropy = 0.339189
Epoch 238
Validation binary_cross_entropy = 0.334432
Epoch 239
Loss = 3.6304e-01, PNorm = 40.5823, GNorm = 3.1759, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.346189
Epoch 240
Validation binary_cross_entropy = 0.374470
Epoch 241
Validation binary_cross_entropy = 0.399720
Epoch 242
Validation binary_cross_entropy = 0.419106
Epoch 243
Validation binary_cross_entropy = 0.430501
Epoch 244
Loss = 3.1097e-01, PNorm = 40.6180, GNorm = 2.1496, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.384717
Epoch 245
Validation binary_cross_entropy = 0.357900
Epoch 246
Validation binary_cross_entropy = 0.342530
Epoch 247
Validation binary_cross_entropy = 0.345445
Epoch 248
Validation binary_cross_entropy = 0.374681
Epoch 249
Loss = 3.0161e-01, PNorm = 40.6544, GNorm = 2.1008, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.394643
Epoch 250
Validation binary_cross_entropy = 0.403401
Epoch 251
Validation binary_cross_entropy = 0.406509
Epoch 252
Validation binary_cross_entropy = 0.395185
Epoch 253
Validation binary_cross_entropy = 0.370038
Epoch 254
Loss = 2.1181e-01, PNorm = 40.6878, GNorm = 1.9554, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.372831
Epoch 255
Validation binary_cross_entropy = 0.384310
Epoch 256
Validation binary_cross_entropy = 0.401962
Epoch 257
Validation binary_cross_entropy = 0.425049
Epoch 258
Validation binary_cross_entropy = 0.415322
Epoch 259
Loss = 2.6909e-01, PNorm = 40.7196, GNorm = 2.3538, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.394941
Epoch 260
Validation binary_cross_entropy = 0.355075
Epoch 261
Validation binary_cross_entropy = 0.342372
Epoch 262
Validation binary_cross_entropy = 0.333998
Epoch 263
Validation binary_cross_entropy = 0.340871
Epoch 264
Loss = 4.6347e-01, PNorm = 40.7517, GNorm = 3.8527, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.371241
Epoch 265
Validation binary_cross_entropy = 0.428661
Epoch 266
Validation binary_cross_entropy = 0.467032
Epoch 267
Validation binary_cross_entropy = 0.462428
Epoch 268
Validation binary_cross_entropy = 0.415294
Epoch 269
Loss = 3.4359e-01, PNorm = 40.7821, GNorm = 2.6666, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.371810
Epoch 270
Validation binary_cross_entropy = 0.347515
Epoch 271
Validation binary_cross_entropy = 0.341675
Epoch 272
Validation binary_cross_entropy = 0.348625
Epoch 273
Validation binary_cross_entropy = 0.353661
Epoch 274
Loss = 3.4659e-01, PNorm = 40.8173, GNorm = 2.7201, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.369646
Epoch 275
Validation binary_cross_entropy = 0.383422
Epoch 276
Validation binary_cross_entropy = 0.376420
Epoch 277
Validation binary_cross_entropy = 0.357081
Epoch 278
Validation binary_cross_entropy = 0.340192
Epoch 279
Loss = 2.8994e-01, PNorm = 40.8518, GNorm = 2.5392, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.336309
Epoch 280
Validation binary_cross_entropy = 0.347565
Epoch 281
Validation binary_cross_entropy = 0.364571
Epoch 282
Validation binary_cross_entropy = 0.364675
Epoch 283
Validation binary_cross_entropy = 0.383870
Epoch 284
Loss = 3.0826e-01, PNorm = 40.8855, GNorm = 2.4692, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.393250
Epoch 285
Validation binary_cross_entropy = 0.395326
Epoch 286
Validation binary_cross_entropy = 0.409784
Epoch 287
Validation binary_cross_entropy = 0.443771
Epoch 288
Validation binary_cross_entropy = 0.465502
Epoch 289
Loss = 3.7686e-01, PNorm = 40.9169, GNorm = 2.3303, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.444308
Epoch 290
Validation binary_cross_entropy = 0.419407
Epoch 291
Validation binary_cross_entropy = 0.372311
Epoch 292
Validation binary_cross_entropy = 0.355506
Epoch 293
Validation binary_cross_entropy = 0.347462
Epoch 294
Loss = 2.6379e-01, PNorm = 40.9501, GNorm = 2.0636, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.345373
Epoch 295
Validation binary_cross_entropy = 0.351341
Epoch 296
Validation binary_cross_entropy = 0.361254
Epoch 297
Validation binary_cross_entropy = 0.369364
Epoch 298
Validation binary_cross_entropy = 0.375342
Epoch 299
Loss = 2.1773e-01, PNorm = 40.9813, GNorm = 2.0951, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.357390
Model 0 best validation binary_cross_entropy = 0.290803 on epoch 7
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.238046
Ensemble test binary_cross_entropy = 0.238046
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 1,303,001
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.323397
Epoch 1
Validation binary_cross_entropy = 0.587416
Epoch 2
Validation binary_cross_entropy = 0.496656
Epoch 3
Validation binary_cross_entropy = 0.372816
Epoch 4
Loss = 9.9261e-01, PNorm = 38.3087, GNorm = 9.2411, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.685885
Epoch 5
Validation binary_cross_entropy = 0.378433
Epoch 6
Validation binary_cross_entropy = 0.504992
Epoch 7
Validation binary_cross_entropy = 0.767002
Epoch 8
Validation binary_cross_entropy = 0.420903
Epoch 9
Loss = 8.6917e-01, PNorm = 38.3949, GNorm = 8.8391, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.691052
Epoch 10
Validation binary_cross_entropy = 0.440921
Epoch 11
Validation binary_cross_entropy = 0.354436
Epoch 12
Validation binary_cross_entropy = 0.602304
Epoch 13
Validation binary_cross_entropy = 0.707339
Epoch 14
Loss = 7.6333e-01, PNorm = 38.5121, GNorm = 7.5684, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.552500
Epoch 15
Validation binary_cross_entropy = 0.610617
Epoch 16
Validation binary_cross_entropy = 0.437277
Epoch 17
Validation binary_cross_entropy = 0.386347
Epoch 18
Validation binary_cross_entropy = 0.733412
Epoch 19
Loss = 9.0276e-01, PNorm = 38.6176, GNorm = 10.5931, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.716474
Epoch 20
Validation binary_cross_entropy = 0.289769
Epoch 21
Validation binary_cross_entropy = 0.400823
Epoch 22
Validation binary_cross_entropy = 0.772535
Epoch 23
Validation binary_cross_entropy = 0.599677
Epoch 24
Loss = 6.7092e-01, PNorm = 38.7076, GNorm = 4.6877, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.370068
Epoch 25
Validation binary_cross_entropy = 0.371136
Epoch 26
Validation binary_cross_entropy = 0.505946
Epoch 27
Validation binary_cross_entropy = 0.671025
Epoch 28
Validation binary_cross_entropy = 0.405265
Epoch 29
Loss = 6.5706e-01, PNorm = 38.7809, GNorm = 5.7491, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.348990
Epoch 30
Validation binary_cross_entropy = 0.413126
Epoch 31
Validation binary_cross_entropy = 0.572301
Epoch 32
Validation binary_cross_entropy = 0.577836
Epoch 33
Validation binary_cross_entropy = 0.493807
Epoch 34
Loss = 4.9614e-01, PNorm = 38.8422, GNorm = 3.4346, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.378593
Epoch 35
Validation binary_cross_entropy = 0.453130
Epoch 36
Validation binary_cross_entropy = 0.539167
Epoch 37
Validation binary_cross_entropy = 0.474016
Epoch 38
Validation binary_cross_entropy = 0.408952
Epoch 39
Loss = 6.5422e-01, PNorm = 38.8955, GNorm = 3.8065, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.380325
Epoch 40
Validation binary_cross_entropy = 0.407322
Epoch 41
Validation binary_cross_entropy = 0.534802
Epoch 42
Validation binary_cross_entropy = 0.524378
Epoch 43
Validation binary_cross_entropy = 0.428395
Epoch 44
Loss = 6.5047e-01, PNorm = 38.9448, GNorm = 4.6729, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.372301
Epoch 45
Validation binary_cross_entropy = 0.405578
Epoch 46
Validation binary_cross_entropy = 0.539294
Epoch 47
Validation binary_cross_entropy = 0.740175
Epoch 48
Validation binary_cross_entropy = 0.605031
Epoch 49
Loss = 6.8862e-01, PNorm = 38.9916, GNorm = 5.0788, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.438721
Epoch 50
Validation binary_cross_entropy = 0.413794
Epoch 51
Validation binary_cross_entropy = 0.467916
Epoch 52
Validation binary_cross_entropy = 0.597026
Epoch 53
Validation binary_cross_entropy = 0.638198
Epoch 54
Loss = 4.8915e-01, PNorm = 39.0381, GNorm = 3.5039, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.537185
Epoch 55
Validation binary_cross_entropy = 0.464133
Epoch 56
Validation binary_cross_entropy = 0.467418
Epoch 57
Validation binary_cross_entropy = 0.471875
Epoch 58
Validation binary_cross_entropy = 0.639547
Epoch 59
Loss = 4.9551e-01, PNorm = 39.0798, GNorm = 3.4019, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.915235
Epoch 60
Validation binary_cross_entropy = 0.779814
Epoch 61
Validation binary_cross_entropy = 0.618282
Epoch 62
Validation binary_cross_entropy = 0.502235
Epoch 63
Validation binary_cross_entropy = 0.517543
Epoch 64
Loss = 5.2298e-01, PNorm = 39.1204, GNorm = 2.4909, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.669668
Epoch 65
Validation binary_cross_entropy = 0.848827
Epoch 66
Validation binary_cross_entropy = 0.919360
Epoch 67
Validation binary_cross_entropy = 0.786928
Epoch 68
Validation binary_cross_entropy = 0.647863
Epoch 69
Loss = 5.6989e-01, PNorm = 39.1602, GNorm = 3.3448, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.629241
Epoch 70
Validation binary_cross_entropy = 0.799540
Epoch 71
Validation binary_cross_entropy = 0.981665
Epoch 72
Validation binary_cross_entropy = 0.776360
Epoch 73
Validation binary_cross_entropy = 0.584099
Epoch 74
Loss = 4.4690e-01, PNorm = 39.2025, GNorm = 2.6254, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.490282
Epoch 75
Validation binary_cross_entropy = 0.489989
Epoch 76
Validation binary_cross_entropy = 0.461139
Epoch 77
Validation binary_cross_entropy = 0.474606
Epoch 78
Validation binary_cross_entropy = 0.536087
Epoch 79
Loss = 4.3545e-01, PNorm = 39.2442, GNorm = 2.9381, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.697354
Epoch 80
Validation binary_cross_entropy = 0.790219
Epoch 81
Validation binary_cross_entropy = 0.776998
Epoch 82
Validation binary_cross_entropy = 0.616569
Epoch 83
Validation binary_cross_entropy = 0.530509
Epoch 84
Loss = 4.8101e-01, PNorm = 39.2887, GNorm = 3.3546, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.564796
Epoch 85
Validation binary_cross_entropy = 0.503382
Epoch 86
Validation binary_cross_entropy = 0.500346
Epoch 87
Validation binary_cross_entropy = 0.433935
Epoch 88
Validation binary_cross_entropy = 0.397623
Epoch 89
Loss = 4.3969e-01, PNorm = 39.3311, GNorm = 3.5106, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.367185
Epoch 90
Validation binary_cross_entropy = 0.399527
Epoch 91
Validation binary_cross_entropy = 0.474614
Epoch 92
Validation binary_cross_entropy = 0.501410
Epoch 93
Validation binary_cross_entropy = 0.471237
Epoch 94
Loss = 3.5227e-01, PNorm = 39.3700, GNorm = 2.1112, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.390336
Epoch 95
Validation binary_cross_entropy = 0.343370
Epoch 96
Validation binary_cross_entropy = 0.366683
Epoch 97
Validation binary_cross_entropy = 0.466068
Epoch 98
Validation binary_cross_entropy = 0.570971
Epoch 99
Loss = 4.6722e-01, PNorm = 39.4143, GNorm = 3.9820, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.480683
Epoch 100
Validation binary_cross_entropy = 0.387278
Epoch 101
Validation binary_cross_entropy = 0.377690
Epoch 102
Validation binary_cross_entropy = 0.389275
Epoch 103
Validation binary_cross_entropy = 0.420338
Epoch 104
Loss = 4.3406e-01, PNorm = 39.4594, GNorm = 2.5120, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.503097
Epoch 105
Validation binary_cross_entropy = 0.549317
Epoch 106
Validation binary_cross_entropy = 0.464518
Epoch 107
Validation binary_cross_entropy = 0.370457
Epoch 108
Validation binary_cross_entropy = 0.348031
Epoch 109
Loss = 4.7129e-01, PNorm = 39.5036, GNorm = 3.5161, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.400944
Epoch 110
Validation binary_cross_entropy = 0.482392
Epoch 111
Validation binary_cross_entropy = 0.508757
Epoch 112
Validation binary_cross_entropy = 0.416415
Epoch 113
Validation binary_cross_entropy = 0.343704
Epoch 114
Loss = 3.7340e-01, PNorm = 39.5433, GNorm = 2.6104, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.350452
Epoch 115
Validation binary_cross_entropy = 0.408733
Epoch 116
Validation binary_cross_entropy = 0.496241
Epoch 117
Validation binary_cross_entropy = 0.524506
Epoch 118
Validation binary_cross_entropy = 0.432058
Epoch 119
Loss = 4.2222e-01, PNorm = 39.5850, GNorm = 3.0018, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.363839
Epoch 120
Validation binary_cross_entropy = 0.343030
Epoch 121
Validation binary_cross_entropy = 0.373956
Epoch 122
Validation binary_cross_entropy = 0.427228
Epoch 123
Validation binary_cross_entropy = 0.490303
Epoch 124
Loss = 3.3266e-01, PNorm = 39.6274, GNorm = 1.9654, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.509672
Epoch 125
Validation binary_cross_entropy = 0.502864
Epoch 126
Validation binary_cross_entropy = 0.419587
Epoch 127
Validation binary_cross_entropy = 0.385943
Epoch 128
Validation binary_cross_entropy = 0.403883
Epoch 129
Loss = 3.6404e-01, PNorm = 39.6733, GNorm = 2.7165, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.439668
Epoch 130
Validation binary_cross_entropy = 0.468896
Epoch 131
Validation binary_cross_entropy = 0.433221
Epoch 132
Validation binary_cross_entropy = 0.386271
Epoch 133
Validation binary_cross_entropy = 0.393164
Epoch 134
Loss = 4.4094e-01, PNorm = 39.7147, GNorm = 4.3821, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.401925
Epoch 135
Validation binary_cross_entropy = 0.397656
Epoch 136
Validation binary_cross_entropy = 0.484214
Epoch 137
Validation binary_cross_entropy = 0.484744
Epoch 138
Validation binary_cross_entropy = 0.412249
Epoch 139
Loss = 3.7227e-01, PNorm = 39.7569, GNorm = 2.7524, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.376695
Epoch 140
Validation binary_cross_entropy = 0.391191
Epoch 141
Validation binary_cross_entropy = 0.445118
Epoch 142
Validation binary_cross_entropy = 0.522100
Epoch 143
Validation binary_cross_entropy = 0.529473
Epoch 144
Loss = 5.0320e-01, PNorm = 39.8021, GNorm = 4.3320, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.394114
Epoch 145
Validation binary_cross_entropy = 0.321835
Epoch 146
Validation binary_cross_entropy = 0.331391
Epoch 147
Validation binary_cross_entropy = 0.394873
Epoch 148
Validation binary_cross_entropy = 0.553452
Epoch 149
Loss = 3.8527e-01, PNorm = 39.8505, GNorm = 3.6949, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.620258
Epoch 150
Validation binary_cross_entropy = 0.550660
Epoch 151
Validation binary_cross_entropy = 0.411644
Epoch 152
Validation binary_cross_entropy = 0.391012
Epoch 153
Validation binary_cross_entropy = 0.434528
Epoch 154
Loss = 4.4914e-01, PNorm = 39.8926, GNorm = 2.9110, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.420893
Epoch 155
Validation binary_cross_entropy = 0.439686
Epoch 156
Validation binary_cross_entropy = 0.521374
Epoch 157
Validation binary_cross_entropy = 0.567827
Epoch 158
Validation binary_cross_entropy = 0.519771
Epoch 159
Loss = 4.2722e-01, PNorm = 39.9360, GNorm = 3.5283, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.383632
Epoch 160
Validation binary_cross_entropy = 0.349316
Epoch 161
Validation binary_cross_entropy = 0.384497
Epoch 162
Validation binary_cross_entropy = 0.497110
Epoch 163
Validation binary_cross_entropy = 0.646218
Epoch 164
Loss = 3.7709e-01, PNorm = 39.9781, GNorm = 3.6478, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.622288
Epoch 165
Validation binary_cross_entropy = 0.514702
Epoch 166
Validation binary_cross_entropy = 0.425937
Epoch 167
Validation binary_cross_entropy = 0.402593
Epoch 168
Validation binary_cross_entropy = 0.470081
Epoch 169
Loss = 3.3682e-01, PNorm = 40.0199, GNorm = 2.6452, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.636174
Epoch 170
Validation binary_cross_entropy = 0.665621
Epoch 171
Validation binary_cross_entropy = 0.522387
Epoch 172
Validation binary_cross_entropy = 0.420504
Epoch 173
Validation binary_cross_entropy = 0.425225
Epoch 174
Loss = 3.4895e-01, PNorm = 40.0620, GNorm = 2.2982, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.464522
Epoch 175
Validation binary_cross_entropy = 0.456889
Epoch 176
Validation binary_cross_entropy = 0.455196
Epoch 177
Validation binary_cross_entropy = 0.432324
Epoch 178
Validation binary_cross_entropy = 0.399159
Epoch 179
Loss = 4.3176e-01, PNorm = 40.1024, GNorm = 3.0534, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.366286
Epoch 180
Validation binary_cross_entropy = 0.349879
Epoch 181
Validation binary_cross_entropy = 0.401526
Epoch 182
Validation binary_cross_entropy = 0.437524
Epoch 183
Validation binary_cross_entropy = 0.514513
Epoch 184
Loss = 4.9733e-01, PNorm = 40.1413, GNorm = 3.4897, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.551455
Epoch 185
Validation binary_cross_entropy = 0.488409
Epoch 186
Validation binary_cross_entropy = 0.519772
Epoch 187
Validation binary_cross_entropy = 0.517485
Epoch 188
Validation binary_cross_entropy = 0.470486
Epoch 189
Loss = 2.7192e-01, PNorm = 40.1819, GNorm = 2.3125, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.457948
Epoch 190
Validation binary_cross_entropy = 0.455455
Epoch 191
Validation binary_cross_entropy = 0.437194
Epoch 192
Validation binary_cross_entropy = 0.411457
Epoch 193
Validation binary_cross_entropy = 0.416425
Epoch 194
Loss = 3.7029e-01, PNorm = 40.2239, GNorm = 2.5780, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.435080
Epoch 195
Validation binary_cross_entropy = 0.438150
Epoch 196
Validation binary_cross_entropy = 0.427680
Epoch 197
Validation binary_cross_entropy = 0.410871
Epoch 198
Validation binary_cross_entropy = 0.415673
Epoch 199
Loss = 3.5292e-01, PNorm = 40.2633, GNorm = 1.9782, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.391940
Epoch 200
Validation binary_cross_entropy = 0.385926
Epoch 201
Validation binary_cross_entropy = 0.394987
Epoch 202
Validation binary_cross_entropy = 0.401867
Epoch 203
Validation binary_cross_entropy = 0.402835
Epoch 204
Loss = 4.1988e-01, PNorm = 40.3026, GNorm = 2.2842, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.369954
Epoch 205
Validation binary_cross_entropy = 0.335555
Epoch 206
Validation binary_cross_entropy = 0.347425
Epoch 207
Validation binary_cross_entropy = 0.397136
Epoch 208
Validation binary_cross_entropy = 0.472012
Epoch 209
Loss = 3.1115e-01, PNorm = 40.3381, GNorm = 2.1319, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.526034
Epoch 210
Validation binary_cross_entropy = 0.475435
Epoch 211
Validation binary_cross_entropy = 0.388265
Epoch 212
Validation binary_cross_entropy = 0.339904
Epoch 213
Validation binary_cross_entropy = 0.336652
Epoch 214
Loss = 3.5229e-01, PNorm = 40.3746, GNorm = 2.8333, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.390802
Epoch 215
Validation binary_cross_entropy = 0.464781
Epoch 216
Validation binary_cross_entropy = 0.512058
Epoch 217
Validation binary_cross_entropy = 0.481757
Epoch 218
Validation binary_cross_entropy = 0.438952
Epoch 219
Loss = 2.7191e-01, PNorm = 40.4124, GNorm = 1.9866, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.427871
Epoch 220
Validation binary_cross_entropy = 0.423586
Epoch 221
Validation binary_cross_entropy = 0.407028
Epoch 222
Validation binary_cross_entropy = 0.410507
Epoch 223
Validation binary_cross_entropy = 0.365710
Epoch 224
Loss = 2.8862e-01, PNorm = 40.4513, GNorm = 2.1512, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.338598
Epoch 225
Validation binary_cross_entropy = 0.334423
Epoch 226
Validation binary_cross_entropy = 0.348346
Epoch 227
Validation binary_cross_entropy = 0.340764
Epoch 228
Validation binary_cross_entropy = 0.351901
Epoch 229
Loss = 2.5494e-01, PNorm = 40.4886, GNorm = 2.8645, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.366398
Epoch 230
Validation binary_cross_entropy = 0.351631
Epoch 231
Validation binary_cross_entropy = 0.355960
Epoch 232
Validation binary_cross_entropy = 0.353537
Epoch 233
Validation binary_cross_entropy = 0.360314
Epoch 234
Loss = 3.7627e-01, PNorm = 40.5226, GNorm = 3.1451, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.364428
Epoch 235
Validation binary_cross_entropy = 0.355172
Epoch 236
Validation binary_cross_entropy = 0.349021
Epoch 237
Validation binary_cross_entropy = 0.371553
Epoch 238
Validation binary_cross_entropy = 0.387194
Epoch 239
Loss = 4.2827e-01, PNorm = 40.5579, GNorm = 3.1318, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.383804
Epoch 240
Validation binary_cross_entropy = 0.414299
Epoch 241
Validation binary_cross_entropy = 0.422772
Epoch 242
Validation binary_cross_entropy = 0.383657
Epoch 243
Validation binary_cross_entropy = 0.369202
Epoch 244
Loss = 3.9681e-01, PNorm = 40.5927, GNorm = 2.8077, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.348506
Epoch 245
Validation binary_cross_entropy = 0.346321
Epoch 246
Validation binary_cross_entropy = 0.357758
Epoch 247
Validation binary_cross_entropy = 0.374339
Epoch 248
Validation binary_cross_entropy = 0.404824
Epoch 249
Loss = 3.6766e-01, PNorm = 40.6290, GNorm = 3.8730, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.399222
Epoch 250
Validation binary_cross_entropy = 0.356984
Epoch 251
Validation binary_cross_entropy = 0.341352
Epoch 252
Validation binary_cross_entropy = 0.330888
Epoch 253
Validation binary_cross_entropy = 0.337240
Epoch 254
Loss = 3.4665e-01, PNorm = 40.6655, GNorm = 2.5388, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.337730
Epoch 255
Validation binary_cross_entropy = 0.360378
Epoch 256
Validation binary_cross_entropy = 0.364711
Epoch 257
Validation binary_cross_entropy = 0.342572
Epoch 258
Validation binary_cross_entropy = 0.311992
Epoch 259
Loss = 2.9372e-01, PNorm = 40.7034, GNorm = 2.7547, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.315557
Epoch 260
Validation binary_cross_entropy = 0.339992
Epoch 261
Validation binary_cross_entropy = 0.369777
Epoch 262
Validation binary_cross_entropy = 0.389403
Epoch 263
Validation binary_cross_entropy = 0.363583
Epoch 264
Loss = 3.4665e-01, PNorm = 40.7413, GNorm = 3.5918, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.337289
Epoch 265
Validation binary_cross_entropy = 0.339633
Epoch 266
Validation binary_cross_entropy = 0.317674
Epoch 267
Validation binary_cross_entropy = 0.316214
Epoch 268
Validation binary_cross_entropy = 0.309704
Epoch 269
Loss = 3.8987e-01, PNorm = 40.7791, GNorm = 3.2898, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.315672
Epoch 270
Validation binary_cross_entropy = 0.350056
Epoch 271
Validation binary_cross_entropy = 0.377000
Epoch 272
Validation binary_cross_entropy = 0.368098
Epoch 273
Validation binary_cross_entropy = 0.351721
Epoch 274
Loss = 2.5999e-01, PNorm = 40.8132, GNorm = 2.2240, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.346704
Epoch 275
Validation binary_cross_entropy = 0.355826
Epoch 276
Validation binary_cross_entropy = 0.360389
Epoch 277
Validation binary_cross_entropy = 0.339648
Epoch 278
Validation binary_cross_entropy = 0.329895
Epoch 279
Loss = 3.2280e-01, PNorm = 40.8473, GNorm = 4.0116, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.324152
Epoch 280
Validation binary_cross_entropy = 0.344429
Epoch 281
Validation binary_cross_entropy = 0.362088
Epoch 282
Validation binary_cross_entropy = 0.356922
Epoch 283
Validation binary_cross_entropy = 0.346940
Epoch 284
Loss = 2.6693e-01, PNorm = 40.8807, GNorm = 2.5787, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.340963
Epoch 285
Validation binary_cross_entropy = 0.329826
Epoch 286
Validation binary_cross_entropy = 0.326093
Epoch 287
Validation binary_cross_entropy = 0.326027
Epoch 288
Validation binary_cross_entropy = 0.323944
Epoch 289
Loss = 3.1689e-01, PNorm = 40.9102, GNorm = 2.8312, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.323383
Epoch 290
Validation binary_cross_entropy = 0.340447
Epoch 291
Validation binary_cross_entropy = 0.355450
Epoch 292
Validation binary_cross_entropy = 0.356325
Epoch 293
Validation binary_cross_entropy = 0.336383
Epoch 294
Loss = 3.4074e-01, PNorm = 40.9398, GNorm = 2.7207, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.324409
Epoch 295
Validation binary_cross_entropy = 0.338192
Epoch 296
Validation binary_cross_entropy = 0.364012
Epoch 297
Validation binary_cross_entropy = 0.391255
Epoch 298
Validation binary_cross_entropy = 0.400942
Epoch 299
Loss = 3.2647e-01, PNorm = 40.9703, GNorm = 2.7101, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.375187
Model 0 best validation binary_cross_entropy = 0.289769 on epoch 20
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.181875
Ensemble test binary_cross_entropy = 0.181875
10-fold cross validation
	Seed 0 ==> test binary_cross_entropy = 0.196439
	Seed 1 ==> test binary_cross_entropy = 0.200195
	Seed 2 ==> test binary_cross_entropy = 0.207248
	Seed 3 ==> test binary_cross_entropy = 0.230162
	Seed 4 ==> test binary_cross_entropy = 0.209275
	Seed 5 ==> test binary_cross_entropy = 0.195285
	Seed 6 ==> test binary_cross_entropy = 0.196908
	Seed 7 ==> test binary_cross_entropy = 0.197607
	Seed 8 ==> test binary_cross_entropy = 0.238046
	Seed 9 ==> test binary_cross_entropy = 0.181875
Overall test binary_cross_entropy = 0.205304 +/- 0.016104
Elapsed time = 0:03:09
